How to Observe LangChain Apps with OpenTelemetry

Observing LangChain apps with OpenTelemetry means instrumenting the framework so every chain, LLM call, tool and retriever becomes a span carrying model, token, cost and latency attributes over OTLP, giving you a full trace of an LLM workflow you own, rather than only what a proprietary tool shows..

Observing LangChain apps with OpenTelemetry means instrumenting the framework so every chain, LLM call, tool and retriever becomes a span carrying model, token, cost and latency attributes over OTLP, giving you a full trace of an LLM workflow you own, rather than only what a proprietary tool shows.

What to capture in a LangChain app

A LangChain application is a graph of steps: chains that call other chains, LLM invocations, tool calls, and, in retrieval apps, retrievers that fetch context. The signals worth capturing map onto that structure: a span per chain and step, and, for each LLM call, the model, prompt and completion tokens (which drive cost), latency and any error or rate-limit response. The value of tracing the whole graph is that a slow or expensive request is rarely the LLM alone; it is a particular chain, a retriever, or a tool call, and only the trace shows which.

How to instrument LangChain

OpenTelemetry has emerging GenAI semantic conventions for LLM and agent spans, and libraries such as OpenLLMetry and OpenInference instrument the popular frameworks to emit them, with token, model and cost attributes, over OTLP. For LangChain specifically, the instrumentation hooks into the framework's callback system, so chains, LLM calls, tools and retrievers are captured automatically once the instrumentation is loaded, with an OTLP exporter pointing at your Collector. LangChain has a native tool in LangSmith, but it is proprietary; instrumenting with OpenTelemetry keeps your traces portable and in the same backend as the rest of your telemetry, so an LLM workflow is observed alongside the services around it.

Prompts and completions can contain personal or sensitive data, so decide deliberately whether to capture their content, and redact or sample it. Keep the token and cost metrics complete even when you sample the bodies.

Key metrics to watch

The signals that explain most LangChain incidents:

What to put on a dashboard

A LangChain dashboard keeps cost and experience side by side. Row one is cost: tokens and estimated cost per request by model, with the per-request token distribution so a creeping prompt size is caught before the bill is. Row two is latency: model call latency and total chain latency, so you can see whether the LLM or the surrounding steps dominate. Row three, for retrieval chains and tools: retriever latency and result counts, and tool call errors.

# tokens per request by model (rising = prompt bloat or bigger outputs) gen_ai.usage.input_tokens + gen_ai.usage.output_tokens by (gen_ai.request.model) # chain latency vs model latency (is the LLM or the chain slow?) histogram_quantile(0.95, chain.span.duration) by (chain.name) # rate-limit (429) rate by model rate(gen_ai_errors{type="rate_limit"}[5m]) by (gen_ai.request.model)

Reading the signals: common failure modes

Cost spike from prompt bloat

Spend jumps without a matching jump in traffic, because tokens per request grew, a prompt template expanded or context is being stuffed. The per-request token distribution reveals it, and the trace of one expensive request shows which chain or retriever ballooned the context. Trim prompts and cap retrieved context.

A slow step, not a slow model

End-to-end latency rises but the LLM call is fast; the time is in a retriever, a tool, or a nested chain. The chain span breakdown points at the exact step, which a model-only view would miss entirely.

Rate-limit errors

429s climb as you approach the provider's tokens-per-minute limit, and requests start failing or retrying. The rate-limit rate by model is the signal. Spread load, request higher quota, add backoff, or route overflow to a secondary model.

Example alert conditions

Starting thresholds to tune:

From monitoring to autonomous resolution

Opstral's AI Ops pillar operates LangChain apps in production, ingesting this OpenTelemetry data, watching cost, latency and retrieval quality, and taking governed action when a chain or model degrades. Explore AI Ops and Telemetry Ops.