AO.com uses APM to diagnose, investigate, and treat incidents

In an English pub in 2000, when a friend of John Roberts bet him £1 that he couldn’t sell white goods—AKA home appliances such as ovens, fridges, and dishwashers—over the internet, Roberts vowed to prove his mate wrong. How? By establishing Appliances Online, which in 2013 rebranded as AO.com. Known for its customer service and slick product review videos, produced in-house, AO.com now offers more than 5,000 products. Those products have expanded to include other essential home electronics such as small appliances, TV and audio systems, laptops, desktops, and computing accessories.

Competing on experience

As a retailer making the majority of its sales through its e-commerce platform, AO.com knows that solid IT is crucial to its ongoing success. For Director of Systems Engineering Carl Phillips, who oversees 10 developer teams, that means also taking responsibility for various ERP and logistics systems, most of which are built in-house. But Phillips’ main focus is AO.com’s customer-facing website. "We compete by providing amazing customer service and on experience and convenience—how simple and enjoyable it is to use our website," he says. "That means we have to ship new features or optimizations as fast as we possibly can—certainly faster than any of our competitors."

For Phillips, the performance monitoring data provided by New Relic is absolutely key to maintaining that competitive edge. 

Data-rich and ready to scale

Like many organizations that have experienced rapid growth, AO.com ran into some issues related to scalability. "As we matured as an engineering organization, we had a lot of hand-rolled instrumentation that just wasn't scaling with us," Phillips says. Scaling up required keeping better tabs on how the company’s various systems were performing. Having tried and rejected other monitoring tools, Phillips was instantly impressed by what New Relic could offer. "The first time I logged into it, I was just amazed by the richness of the data. I knew straight away that we'd found what we needed."

Maintaining an edge through monitoring

Since day one, New Relic application performance monitoring (APM) has been AO.com’s go-to tool. "It’s our de facto standard, so we use it everywhere," Phillips says. For diagnosing, investigating, and treating incidents, New Relic is invaluable. Phillips and his teams also use it for regular load tests, as well as for assessing the scalability profiles of web apps. "Measuring things like the percentiles, the median, the mean, the 95th percentile for page-load time—and using them as a basis for improvement—has been really important to us."

AO.com also uses synthetics, which the company uses for smoke tests and alert-type monitoring on its main site. "Using synthetic monitoring for measuring availability has helped us to create some really nice KPIs around Apdex and availability," Phillips says. "Any performance improvements we make, we can see translate into the synthetic scores." Those scores become even more important at peak times, such as during Black Friday and Boxing Day sales. synthetics allow AO.com to understand if they have any problems on any core parts of the customer journey, or if performance degradations are due to a customer’s device or connection speed because they can simulate tests repeatedly from the same location. 

Converting and customizing

Phillips finds the customizable instrumentation of dashboards super useful in his mission to expand AO.com’s data-driven decision-making beyond the IT department. "I've got quite a few commercial analysts now who are using New Relic," he says. "They’re building dashboards of their own and making ad hoc queries to find out what's going on." By creating unique dashboards focused on conversion funnels, analysts have achieved a deeper understanding of the conversion pipeline—particularly how conversion is impacted by performance. And it’s not just commercial analysts Phillips’ converted into New Relic cheerleaders. "I’ve found very few barriers to people using it. I love that I can just give it to businesspeople. I can tell them, 'Don’t be afraid. You can't break anything,' and just let them go and play."

For Phillips, showing his colleagues the New Relic ropes calls to mind that old adage about giving a man a fishing rod, not just a fish.  "In a meeting once, someone asked me a question related to data. So, I just wrote a little query and showed them the results in charts. And they were like, 'What’s that? That’s really good!’"

Automating and transitioning

Phillips is proud that AO’s developers have always had operations responsibilities, too. And the company’s commitment to automation reflects an embracing of the latest trends in the world of software. "Continuous integration, automated testing, fast feedback loops—all the concepts that DevOps and Agile promotes, we embrace." New Relic is integral to such strategies. "We instrument our deployments automatically into New Relic,” he adds, "so we use it as part of every deployment."

Another trend that Phillips is following closely is the move towards cloud-based infrastructure. "We’re definitely approaching the tipping point where we'll put our e-commerce platform onto public cloud, and then there'll be no going back." The transition, he admits, will involve a steep learning curve. He’s excited about the ability to hook their cloud provider directly into New Relic dashboards, generating insights that can monitor storage growth. "If the forecast really changes, we can spot that, and we can actually tie it back to cause and effect so we know we're making responsible decisions."

He’s also impressed by the option, offered by New Relic infrastructure, to filter by availability zone. "Knowing straightaway that a particular server in your farm is down can really help you find a solution, fast." In other words, as AO continues to evolve, New Relic is like the white goods in a home kitchen—essential.