Southern Cross Austereo slashes open source costs by 50%

Southern Cross Austereo (SCA) is one of Australia’s leading media companies reaching more than 95% of the Australian population through its radio, television, and digital assets. Under the Triple M and Hit network brands, SCA owns 99 stations across FM, AM, and DAB+ radio. SCA provides national sales representation for 34 regional radio stations and broadcasts 93 free to air TV signals across regional Australia. It also operates LiSTNR, Australia’s free audio destination for consumers, housing radio, podcasts, music, and news.

Challenge: Scaling while manually troubleshooting

Like many industries, Australia’s radio broadcasting sector is going increasingly digital. Faced with competition from subscription services like Spotify and Apple Music, radio broadcasters are looking for ways to innovate and increase audience share. As part of a digital-first initiative, SCA created a major new application, dubbed LiSTNR, for its audience. Developed by Australians for Australians, the app is designed to consolidate all of SCA’s music, radio & podcasts under one umbrella.

SCA knew it needed an understanding of what its users were experiencing, but this was hampered by a lack of observability. Application visibility was lacking in root cause analysis, leaving engineers to manually troubleshoot, with tracing being more complicated when performing manual tasks.

SCA had previously looked to open source tools for monitoring but these were becoming increasingly difficult to use.

"Over a period of time, the tools themselves became quite complex to manage and maintain, to keep the system up and running, and ensure that there were no issues. The main problem was that if we ever had an outage on the operational front of our monitoring tool, and then an outage on production software such as LiSTNR, we wouldn't have any visibility over the issues," explains Amit Dsouza, Digital DevOps Engineer, Southern Cross Austereo.

Solution: Visibility through alerts and dashboards

To overcome the lack of visibility, SCA thoroughly researched the market and looked at multiple providers. It finally identified New Relic as ideal for its needs.

"New Relic has the best support and technical prowess. We have tried other tools from similar companies and had problems with either support or with the toolset itself which is why we made the switch," Amit says.

SCA found implementation very easy, especially with the support received from the New Relic team. While only managed as a side project due to different priorities in the business, everything was up and running within a month.

"New Relic was very helpful and they were constantly present for any of our questions. We would have weekly or biweekly calls based on our requirements, and they would either help us physically or point us in the right direction of who could help. So, it was quite smooth sailing overall," Amit says.

SCA set up a standard dashboard in its network centre that shows the current status of all its applications. “It’s a simple display that shows whether an app is up or down, and isn’t overwhelming in terms of detail so the first level support team can quickly trigger an on-call,” Amit explains.

Specific New Relic dashboards have also been set up, as have New Relic alerts for certificate expirations in case auto-refresh stops working. Future dashboards are also planned for the leadership team, in order to provide a clear view of the business impact of technology initiatives and to enable real-time decision-making at the executive level.

New Relic alerts have also allowed peak periods to be managed much easier. “If we have additional users jumping on, auto-scalers are in place to automatically scale up the entire service. But if New Relic alerts show that particular machines are overwhelmed, the team can add servers to the entire Kubernetes setup, and that way prevent any outage during a peak,” he says.

Impact: Rapid growth in users, productivity

Since going live, LiSTNR's radio and podcast streaming has grown significantly. The volume of content consumed per signed-in user has also grown equally, which is reflective of users seeing value in the app.

New Relic alerts have also led to significant productivity gains. By setting up alerts for certain shows so that any problems can be identified the day before the show is due to air, engineers are able to immediately jump on an issue and fix or patch problems. This has greatly improved customer experience, as the team is now the first, instead of the last, to know about any problems.

"Putting alerts into place has really helped a lot. We get instantly alerted about issues and can fix them straight away, rather than waiting until end users are impacted and registering complaints.

"Developers are very happy because New Relic has enabled them to be proactive and patch a particular issue prior to it escalating and affecting users. Leadership is also happy just seeing that they are not the first to have to come back to us with an error in the first place. And the user experience is greatly enhanced as a result, which was our primary aim in deploying an observability solution," Amit says.

Overall, SCA has seen a 20-40% reduction in the time spent to detect, understand and repair issues. Open source hosting, maintenance, and support costs have also been cut by 50%.

The solution is also driving metrics around business strategy. Having full visibility has enabled SCA to report on engineering effort and efficiency against application availability and up-time.