What is observability?

Observability is proactively collecting, visualizing, and applying intelligence to all of your metrics, events, logs, and traces—so you can understand the behavior of your complex digital system.

A simple way of describing observability is how well you can understand the system from the work it does. In control theory, observability is defined as how engineers can infer the internal states of a system from knowledge of that system's external outputs. Expanded to IT, software, and cloud computing, observability is how engineers can understand the current state of a system from the data it generates. To fully understand, you’ve got to proactively collect the right data, and then visualize it and apply intelligence.

A common abbreviation for observability is o11y, which replaces the 11 letters between o and y with the number 11. (Fun fact: That’s why we get k8s for Kubernetes!)

Observability gives engineers a proactive approach to optimizing their systems. It provides a connected real-time view of all the operational data in your software system, as well as the flexibility to ask questions on the fly about your applications and infrastructure to get the answers you need.

Why is observability important?

Modern-day systems are transforming into complex, open source, cloud-native microservices running on Kubernetes clusters. They’re being developed and deployed faster than ever—by distributed teams. With DevOps, continuous delivery, and agile development, the whole software delivery process is faster than ever before, which can make it more difficult to detect issues when they arise.

When things went wrong in the days of mainframes and static operations, it was pretty easy to understand why, and pre-configured static dashboards alerted an operator of an issue. These systems failed in similar ways over and over again.

As systems became more complex, monitoring tools attempted to shed light on what was happening with software performance. You could trace application performance with monitoring data and time-series analytics. It was a manageable process.

Today, the possible causes of failure are abundant—and can feel infinite when you are staring at a screen, frustrated. Is a server down? Is your cloud provider having an outage? Did someone push new code that's impacting end-user behavior?

When working on these complex, distributed systems, identifying a broken link in the chain can be nearly impossible without an observability solution. Now that microservices architectures are commonplace, responsibilities are distributed across teams. There’s not a discrete app owner, and many teams need to be involved. Teams need to understand, analyze, and troubleshoot application areas they don’t necessarily own. You need distributed tracing, which allows you to trace requests—and bottlenecks—through all parts of a distributed system.

Observability vs monitoring

Conventional monitoring won’t help you succeed in the complex world of microservices and distributed systems. It can only track known unknowns. These are the things you know to ask about in advance (for example: “What’s my application’s throughput?”, “What does compute capacity look like?”, “Alert me when I exceed a certain error budget.”) Observability gives you the power to not just know that something is wrong…but to also understand why. It gives you the flexibility to understand patterns you hadn’t even thought about before, the unknown unknowns.

Think of it this way: Observability (a noun) is the approach to how well you can understand your complex system. Monitoring (a verb) is an action you take to help in that approach. Observability doesn't eliminate the need for monitoring. Monitoring just becomes one of the techniques used to achieve observability. 

Application performance monitoring (APM) is one of the steps in a well-rounded observability practice that uses dashboards and alerts for known or expected failures. To learn why it's important to have APM as part of your observability practice, see APM vs. observability.

What are the components of observability? 

Observability in digital systems has four fundamental pieces:

1. Open instrumentation. Instrumentation is using code (agents) to track and measure data flowing through your software application. Open instrumentation means gathering telemetry data from open source or vendor-specific entities that produce that data. Examples of telemetry data include metrics, events, logs, and traces, often referred to as MELT. Examples of entities include services, hosts, applications, and containers.
2. Correlation and context.  Understanding the bigger picture is vital, especially for large enterprise applications with enormous amounts of raw telemetry data. The telemetry data collected must be analyzed for correlations and context, so humans can make sense of any patterns and anomalies that arise.
3. Programmability. Organizations need the flexibility to create their own context and curation with custom applications based on their unique business objectives.
4. AIOps tools. To ensure that your modern infrastructure is always available, you need to accelerate incident response. AIOps solutions use machine learning models to automate IT operations processes such as correlating, aggregating, and prioritizing incident data. These tools help you eliminate false alarms, proactively detect issues, and accelerate mean time to resolution (MTTR).

What are the benefits of observability?

Observability tools empower engineers and developers to create better customer experiences despite the increasing complexity of the digital enterprise. With observability, you can collect, explore, alert, and correlate all telemetry data types.

Observability makes it easier to drive operating efficiencies and fuel innovation and growth. For example, a team can use an observability platform to understand critical incidents that occurred and proactively prevent them from recurring. This decreases downtime and improves MTTR.

When a new build is pushed out, they can see into the application performance and then drill down into the reasons why an error rate spikes or application latency rises. They can see which particular node has the problem. For more examples, organized into 10 principles of observability, see Observability: A 21st Century Manifesto.

Other benefits of observability include:

• A single source of truth for operational data.
• Verified uptime and performance. 
• An understanding of the real-time fluctuations of your digital business performance.
• Better cross-team collaboration to troubleshoot and resolve issues faster.
• A culture of innovation.
• Greater operating efficiency to produce high-quality software at scale, accelerating time to market.
• Specific details to make better data-driven business decisions, and optimizing investments.

The 2022 Observability Forecast found that nearly half of the 1,600+ respondents cited the increased focus on security, governance, risk, and compliance as the main trend driving observability needs for their organization. Other drivers included development of cloud-native application architectures (frontend), increased focus on customer experience management, and migration to a multiple-cloud environment (backend). 

The report also found that only 2% of respondents indicated that their organizations had employed all 15 mature observability practice characteristics, which include automated instrumentation, automated portions of incident response, infrastructure that is provisioned and orchestrated using automation tooling, telemetry captured across the full stack, and telemetry (metrics, events, logs, and traces) unified in a single pane for consumption across teams.

Observability is essential, but there's a lot of room for most businesses to improve.

Who uses observability?

SREs and IT Operations teams are in charge of keeping complex systems—the apps that people rely on every day—up and running. But observability is everyone’s concern throughout the software development lifecycle. 

Software engineering teams use observability to understand the health, performance, and status of software systems, including when and why errors occur. By looking at a system's outputs, such as events, metrics, logs, and traces, engineers can determine how well that system is performing.

Observability and DevOps

Deployment frequency has increased dramatically with microservices. Too much is changing to realistically expect teams to predefine each and every possible failure mode in their environments. It's not just application code, but the infrastructure that supports it, and consumer behavior and demand. 

Observability gives DevOps teams the flexibility they need to test their systems in production, ask questions, and investigate issues that they couldn’t originally predict.

Observability helps DevOps teams:

• Establish clear service-level objectives (SLOs) and put instrumentation in place to prepare and join forces toward measurable success.
• Rally around team dashboards, orchestrate responses, and measure the effects of every change to enhance DevOps practices.
• Review progress, analyze application dependencies and infrastructure resources, and find ways to continually improve the experience for the users of their software.

For DevOps best practices, check out the DevOps Done Right ebook.