Executive Summary
Telematics company, Fleet Complete, has one overarching goal: to grow its business rapidly by using Internet of Things (IoT) devices to collect the data and deliver the insights needed to keep customer commercial vehicle forces rolling and to drive its connected commercial vehicle platform. To sustain this growth, the company has adopted an aggressive acquisition strategy that has resulted in massive influxes of data. Needing an environment that could scale dramatically, the company turned to the AWS cloud and employed New Relic monitoring to facilitate the migration and ensure seamless performance. Within 12 months, the company’s cloud migration was 60 percent complete, its DevOps software release cycle had accelerated from once a month to once a week, and it had gained complete visibility into its all-important data ingestion pipeline.
Driving Hyper-Growth
Serving more than 500,000 subscribers and 35,000 businesses in 17 countries, while experiencing tenfold growth over the last several years, Fleet Complete is the fastest-growing telematics provider in the world today.
Aiming to grow the business by another factor of 10 over the next decade, the company knows that it will need to collect and process an enormous amount of data if it is to meet that goal. Since the more fleet information the company can collect, the more value (in terms of insights) it can deliver to customers.
Explains Fleet Complete Chief Technology Officer, Alan Fong, “It’s not just about collecting data; it’s about layering that data to provide the best insights to customers. When you start layering different types of data (such as GPS data, vehicle health, and more), you can figure out all sorts of things—like how driving behavior affects vehicle brake pads, or how potholes (and the speed at which they are encountered) affect vehicle components and life span.”
To collect and process this massive influx of data (much of it stemming from acquired companies), Fleet Complete built a data ingestion pipeline, which it initially ran from its data center. Over time, however, it became clear that an on-premises solution simply could not keep pace with the growing number of data points the company was collecting. For that, Fleet Complete needed massive scalability that only the AWS cloud could provide.
AWS: A Global Platform for a Global Company
Charged with migrating Fleet Complete’s technology platform to the cloud as quickly and efficiently as possible, Fong turned to AWS.
“I understood the breadth of the AWS tools and the global presence of the AWS platform,” Fong says. “AWS would allow us to use machine learning and AI models atop our own IP to improve the information we deliver to customers. And it would scale to allow us to ingest more and more data.”
New Relic: Full-Platform Visibility to Keep the Data Flowing
Choosing a solution that would provide a single unified and holistic view into the application and infrastructure performance of Fleet Complete’s cloud-based platform, however, proved slightly more difficult. This was largely due to the complexity of an environment that had grown through acquisition and contained a multitude of disparate systems. While Fleet Complete took a hard look at both the monitoring solutions that were already in place, and some that were new to the company, it quickly chose New Relic in proof-of-concept testing.
Explains Fleet Complete Architect Muhamad Samji, “New Relic gave us all the insights we needed—both globally and into the different pieces of our distributed application. For example, we can see what's going on with the host, as well as what's happening with all the applications that reside on that host. So instead of a closed-off view that only reveals that an app is running slowly, New Relic provides visibility straight through to the host, so that we can understand why that app is running slowly. In addition, as we move data across our distributed system, New Relic enables us to see where bottlenecks are occurring, as we call from service to service.”
Solving Tough Technical Challenges with New Relic
Today, Fleet Complete is using both New Relic APM and New Relic Infrastructure to run their services, and both have become essential to running Fleet Complete’s business.
New Relic APM provides critical insights into application performance. As part of Fleet Complete’s data processing pipeline, a series of enrichment steps occur. During these steps, microservices may communicate with other Fleet Complete microservices or external services. APM allows the Fleet Complete team to troubleshoot throughput, error, and latency challenges, connected with these services, and to quickly diagnose issues within the pipeline—using not only application traces and metric data generated by APM, but also New Relic Distributed Tracing.
All of these services, though, rely on a self-managed Kubernetes cluster on Amazon EC2. To troubleshoot issues and understand cluster health, available capacity, and scaling, the Fleet Complete team turned to New Relic Infrastructure and the Kubernetes cluster explorer to gain an at-a-glance understanding of overall cluster health. With this knowledge, they can then drill down further, if needed. The cluster explorer also allows software engineers to quickly understand whether application performance problems identified by New Relic APM are the result of application behavior, or whether they could be caused by infrastructure constraints. Indeed, with just a few clicks, an engineer can navigate from a particular application to an unhealthy pod, and then navigate to trace spans that occurred within it.
“This cluster has a massive amount of data flowing in and out of it,” explains the Fleet Complete team, “The New Relic Kubernetes cluster explorer lets us see host-level data such as IO, memory, and so on. Even more importantly, it makes all of the application pods (like those for our databases) first-class citizens.”
Accelerating a Smooth Migration to the Cloud
A year after Fleet Complete began its migration to the cloud, more than 70% of its environment had been shifted to AWS—an accelerated transition that New Relic played a huge part in facilitating.
Explains Fong, “We didn’t want to disrupt the business, as we migrated to the cloud, so having New Relic APM and New Relic Infrastructure monitoring, installed across both our on-premise and cloud environments was crucial to the process.”
Samji agrees: “Being able to load-test on-prem and then use New Relic to ensure that we’re hitting those same numbers in the cloud (and not creating any hidden bottlenecks) has been key to our smooth and swift migration.”
Indeed, with the unified and holistic insights into application and infrastructure performance provided by New Relic, the risk of business disruption during cloud migration was virtually eliminated. That in turn allowed Fleet Complete to accelerate the migration process and realize the business gains sooner— without adversely affecting service reliability or brand image.
A Seamless Partnering of Solutions
Using Amazon Elastic Compute Cloud (Amazon EC2) instances for the Web- app data ingestion points (or listeners) and Amazon Simple Storage Service (Amazon S3) to house the “lake” of data it amasses, Fleet Complete is able to absorb an ever-increasing amount of data in a process that Fong describes as follows:
“Device data comes into our listeners, which then dump that data into a stream, where it gets enriched at various points. As data moves through that stream, it's also being pushed into a batch workflow, which speeds into our data lake. On the other end of that stream, the data makes its way out to our application—to serve our Web and mobile apps and our APIs.”
At every step in this journey, Fleet Complete uses New Relic APM and New Relic Infrastructure to monitor performance.
New Relic has also helped Fleet Complete understand AWS usage, allowing the DevOps team to take the Amazon-provided Cost Explorer list of underutilized servers, and then use New Relic to see what’s actually going on in those servers, so the company can make optimal use of them.
For Jeff Hobeck, Principal Architect for Fleet Complete, the seamless way in which New Relic works with AWS has been the icing on the cake. “We can see the AWS information we need in New Relic, and it involves very little setup. It's almost like New Relic is native to AWS.”
