Trainline accelerates release cycles while reducing errors by 95%

In 2013, Darren Parsons was brought on to start a new team tasked with re-engineering the frontend of Trainline's consumer website. “One of the things we quickly realized was that we needed to accelerate the feedback cycle in our release strategy,” says Parsons. “Rather than spend 6-12 weeks working on a release, we needed to ship much more frequently. And in order to do that, we needed to start doing continuous delivery.”

Moving to continuous delivery

For continuous delivery to work, however, Parsons’ team would need a way to know what impact their deployments were making on the user experience. Particularly, they wanted a performance monitoring solution that would let them visualize customer impact during and following a release and also make more informed product decisions. Trainline had two primary monitoring tools at the time, neither of which provided the type of real-time insight they were looking for. Says Parsons, “With one tool, the feedback cycle wasn’t quick enough. With the other, we’d need to put in a lot of work on our part to get what we wanted—essentially building a monitoring solution ourselves. So we looked outside to see what we could use.”

Since this was a pilot project for Parsons’ team, they didn’t want a solution that required any capital expenditure. The only tools they would even consider were those that offered a monthly payment model, i.e., operational expenditure. “At the time, there wasn’t really much in the market except for New Relic,” says Parsons. “So we did a very quick installation and within 2-3 hours we were already seeing meaningful numbers coming back to us.” Once Parsons rolled out New Relic to his team, it didn’t take long for Trainline chief technology officer Mark Holt to take notice. “The big thing I saw New Relic giving us was true monitoring of end-user behavior that tied all the way through server performance and down into individual calls,” he says. “We could drill into the line of code that’s causing a performance problem and fix it immediately, which is something we were never able to do before.”

Monitoring the conversion funnel in real time

Soon enough, Trainline's entire technology organization started using New Relic for everything from application, server, browser, and mobile monitoring to real-time business analysis with application performance monitoring, mobile monitoring, and browser monitoring. “Now we can actually see the responsiveness of various aspects, what the conversion funnel looks like,” says Parsons. “We prioritize much of our backlog off of what we see from data rather than supposition. The New Relic platform gives us a data-driven approach to knowing what the problem is.”

“From being able to track individual pages to creating funnels to show where customers are dropping out, there’s a lot of great functionality in dashboards,” says Holt, admitting that he’s “slightly obsessed with it. Every morning, the first thing I do is pick up my phone and check each site’s overnight performance and review where we are in real-time that morning. Slightly embarrassingly, it’s also the last thing I do at night. I tend to bounce between Insights and APM quite a lot.”

For David Stanley, head of IS operations at Trainline, the accountability that New Relic brings into the organization is invaluable. “We’ve been changing our development teams around to give them more ownership,” he says. “So by arming them with New Relic, we can all know when things fail after a release and roll it back immediately.” Both the operations and development teams at Trainline agree that New Relic is a key tool in enabling the DevOps environment they’re striving for.

Trainline has achieved some major wins since using New Relic. For example, the company reduced the number of error pages that customers see by 95%, all within two months of adopting the tool. Not only that, it improved the customer experience of its mobile app significantly. “Thanks to New Relic, we noticed that there was a misconfiguration in the mobile application stack,” explains Holt. “So we tweaked the configuration setting and it immediately dropped the data transfer rate by 87%. For our customers, that means no more sitting and staring at a blank screen to load.”

New Relic has also helped Trainline achieve a more agile development cycle. “With the continuous delivery model that we’ve put in place using New Relic, we’ve been able to accelerate release cycles from weeks to just hours,” says Parsons. “Rather than wait 12 weeks to release one small improvement, we can now roll out around 50 releases within the same time. And that all adds up to a better product for our customers.”

In the near future, Trainline plans to move out of its current data center and migrate its applications to the cloud. “We believe that New Relic will be a fabulous tool for helping us migrate to the cloud successfully,” says Holt, citing the challenges that come with running an app in both a public and private environment. “We can use it to make sure we have a good performance baseline before the move and help ensure that no issues pop up during the migration.”

The end goal of all of this, of course, is to create a better experience for Trainline’s customers. “Our mission is to help rail passengers save money, time, and hassle,” says Holt. “New Relic helps us do that by allowing our developers to see and feel the pain of a customer who’s dealing with a problem on our site. It connects our team to the real people using our product, and I find that incredibly powerful.”