Preview|DDOG 2Q25:LLM Observability, OpenAI and 2Q checks
Solid NT performance and exciting LT opportunities, but wary of OpenAI migration
In our previous note, we discussed the potential of Datadog's agent product Bits AI, and why the product is important for not only DDog's future growth as well as stock narrative. In this note, we will elaborate on how DDog's LLM Observability could be a substantial driver in the future as well. Also, we present our thoughts on DDog's relationship with OpenAI, how it affects upcoming quarters, as well as recent channel checks on 2Q results.
Why LLM Observability is Important
Datadog launched its LLM Observability in mid-2024 as an extension of its Application Performance Management (APM) offerings, specifically to monitor AI applications and agents. Due to hallucinations in AI applications and the non-deterministic nature of agent work, this product has become a crucial component in the AI application inference process.
The product covers input/output metadata, timing information, and error context for each agent action. This granular detail is critical for quickly troubleshooting "soft failures" beyond hard errors, such as an agent selecting the wrong tool, a tool returning incomplete or unexpected data, or a model hallucinating or misunderstanding a prompt. This tracing capability allows developers to gain a deep understanding of the entire agent workflow.
A unified dashboard displays LLM and agent runtime KPIs, including token usage, latency, error rates, and call counts, to help engineers optimize costs.
Traditional APM tools are designed for deterministic, single-agent systems with predictable workflows, while Agentic applications feature non-deterministic outputs and routing, model-based decisions, and dynamic adaptation. This creates a need for purpose-built agentic observability that can handle autonomous, reasoning-based multi-agent systems.
