Gregg Ostrowski, CTO Advisor at Cisco AppDynamics, discusses the technology organizations need to optimize application availability and performance, why IT teams need to convert raw telemetry data into actionable insight and adopt a hybrid strategy which integrates cloud native and on premise observability.
Adoption of OpenTelemetry is increasing rapidly as IT teams look to generate visibility into cloud native applications and underlying infrastructure. In fact, most CIOs I speak to tell me that they expect their organization to be using OpenTelemetry within the next two years.
OpenTelemetry is a holistic telemetry open-source standard for metrics,and traces with the expectation to also support logs. It has been adopted and supported by cloud service providers, observability vendors (including Cisco AppDynamics) and end users. Using OpenTelemetry, with just a few lines of code during the development stage, businesses can get real-time data on application performance once in production. OpenTelemetry offers open standard tools for auto instrumentation of applications built in microservices and Kubernetes environments and easy ways to plug into the CI/CD pipeline.
Already, we’re seeing start-up businesses that are embracing OpenTelemetry as an agile and cost-effective way to build observability into their IT estate from day one, through APIs and SDKs. And within larger enterprises, OpenTelemetry is enabling technologists to generate availability and performance data across increasingly fragmented and complex applications. They’re getting visibility into parts of their IT environment that they simply haven’t been able to see, through their traditional, agent-based monitoring tools.
OpenTelemetry is the foundation for performance within cloud native environments
With IT departments facing unrelenting pressure to innovate at speed, technologists are taking advantage of no code and low code platforms to accelerate release velocity. They recognize the flexibility and scale that cloud native technologies offer.
But this shift is dramatically increasing the complexity of application topologies, with organizations deploying thousands of microservices and containers. To put it simply, if you have a single application that is running on premise and you want to bring that app over into a cloud native construct, you need to break the app up into separate entities. That app will then run as a cloud native, microservice-driven application. And so therefore you’ve suddenly gone from instrumenting one application to instrumenting 10, 12 or 20 different microservices. And these microservices are spawning a massive volume of data every second.
This is why OpenTelemetry is so vital, providing visibility into modern IT environments, across the application level and into the supporting digital services (such as Kubernetes). Without it, technologists have no way to identify and resolve performance issues before they impact end user experience.
IT teams need to convert raw telemetry data into actionable insight
However, it’s vital for technologists to ensure they have a way to consume, process and correlate the mass of telemetry data that they will now be generating. OpenTelemetry is superb at collecting individual, isolated pieces of data and stitching these together, but it doesn’t deliver more holistic insight into performance. Put simply, it’s only concerned with generating data; it doesn’t help technologists to make sense of the vast volumes of data it creates.
On its own, it’s very hard for IT teams to derive any substantial value from OpenTelemetry because it is so chatty, spawning massive volumes of data and making it difficult to visualize the full application picture. This is where IT teams need additional tools to enhance the data they get from OpenTelemetry and combine it with other data points that they derive through agents and data coming in from public cloud environments, such as Kubernetes.
Indeed, for all of its strengths, OpenTelemetry simply doesn’t deliver the same level of visibility as a proprietary agent within on-premise environments. With OpenTelemetry, IT teams still only get traces and metrics.
Organizations need to correlate OpenTelemetry data with business transactions to optimize digital experiences
Critically, OpenTelemetry does not understand business transactions so IT teams need an observability platform that is able to extract business transactions from OpenTelemtry data, while also traversing agent-based monitoring within on-premise environments.
Many IT teams are deploying separate tools to monitor cloud native and legacy applications, and this means they are having to run a split screen mode, with no clear line of sight of the entire application path, where application components are running across on-premise and cloud native environments.
This makes it incredibly hard for IT teams to troubleshoot issues quickly and it drives up metrics such as MTTR and MTTX. Not only that, but the inability to derive business context throughout the overarching application flow makes it impossible for organizations to validate their investments in cloud technologies, something which is becoming increasingly important as organizations look for operational efficiencies.
IT teams therefore need to be able to ingest OpenTelemetry directly into their unified observability platform, to cut through data noise, make informed decisions and prioritize actions based on potential impact to end user experience and the business.
IT teams must adopt a hybrid strategy which integrates cloud native and on premise observability
While cloud native technologies undoubtedly represent the future of computing, the transition is going to take time. Already we’re seeing businesses slowing down on their cloud migration programs due to escalating costs and a backdrop of economic uncertainty. Most organizations are still going to be running the majority of their IT on premise. And therefore, they need a hybrid strategy where they correlate OpenTelemetry into the overarching mix of already instrumented applications through traditional agent-based monitoring systems.
IT teams need an observability platform which provides flexibility to span across both cloud native and on premise environments — with OpenTelemetry and agent-based entities being ingested into the same platform. Only with unified visibility and insights will technologists be able to cut through complexity and optimize application availability and performance at all times.
