Setting the file. One moment.
Subchapter 25.3
OBSERVABILITY-PLATFORMS.mdMarkdown8 KBView on GitHub
Surveyed 2026-10, and not guaranteed to be up to date. Use it to get oriented, then research the current options yourself: products, prices, limits and betas change fast.
Orientation for filling the observability platform slot, or deciding on a deliberate none. The types split by what the human needs to see inside a run: its log, its agent trace (each model call and tool call as a span), or that trace joined to the rest of the user’s system. Self-hosted vs SaaS cuts across the last three types. When a run should act on what the platform sees, make the platform a ticket source: a monitor files the ticket, and a run reads it.
Every type receives data through one of five hooks. Name the hook alongside the product when filling the slot.
CLAUDE_CODE_ENABLE_TELEMETRY=1 plus OTEL_* variables, with prompts and tool output opt-in; Codex CLI takes an [otel] block in config.toml. (Claude Agent SDK (opens in a new tab), Claude Code (opens in a new tab), Codex config (opens in a new tab))configure_claude_agent_sdk().OpenTelemetry is the common wire: harness OTel and platform push land in any OTLP backend. The GenAI semantic conventions (invoke_agent, chat, execute_tool spans) are still settling, so expect span names to drift. (OTel GenAI agent spans (opens in a new tab))
What it is: the human reads each run where the runner or orchestrator already shows it: the CI run page, the sandbox dashboard, the agent’s transcript saved as a run artifact. No hook to build, as self-hosted as the runner, and bundled with its cost.
When a factory picks it: few runs a day, one human watching, and the question is “did it fail, and what did it print?” Most factories start here, and grow out of it when they need search across runs, a token or cost rollup per run, or retention longer than the runner keeps.
Starting points: GitHub Actions run logs (opens in a new tab), E2B sandbox logs and metrics (opens in a new tab), Daytona sandbox telemetry (opens in a new tab), OpenAI Agents SDK traces (opens in a new tab). Sandbox products often forward the same data over OTLP, which is the path out to Type 2.
Verify: retention days per runner product; some sandboxes keep very little (Daytona: 3 days, as of 2026-10).
What it is: a backend built around the agent trace: each run as a tree of model calls and tool calls, with token cost, prompts and sessions, usually with evals and datasets on top.
When a factory picks it: the human asks “why did the agent do that?” across many runs; a retro run or a human compares runs; token spend needs a per-run rollup. Self-hosting an open-source one keeps transcripts in-house.
Traits of the type: takes harness OTel, an SDK wrapper or a hook plugin, often the platform push too. Billed by volume, and agent runs are span-heavy (one span per tool call), so budget per run rather than per request.
Starting points: Langfuse (opens in a new tab) and Arize Phoenix (opens in a new tab) (open source, self-hostable), LangSmith (opens in a new tab), Braintrust (opens in a new tab).
Verify: free-tier limits, the billing unit (trace, span, unit or GB; Braintrust moved to GB in 2026-03), which tier unlocks self-hosting, and current ownership.
What it is: the user’s existing observability stack (logs, metrics, distributed traces for every service), with an agent or LLM view added.
When a factory picks it: the team already pays for one; runs need joining to the app’s own traces (the Agent SDK propagates TRACEPARENT, so a run nests inside its caller’s trace); or a monitor ticket source already reads from it.
Traits of the type: harness OTel goes straight to the vendor’s OTLP endpoint. Priced per GB, event or host, so a factory’s volume is a rounding error on an existing bill and a surprise on a new one.
Starting points: Datadog Agent Observability (opens in a new tab), Grafana Cloud (opens in a new tab), Honeycomb (opens in a new tab), SigNoz (opens in a new tab) (open source).
Verify: whether the agent view is GA or preview, which harnesses have a setup guide, and free-tier volumes.
What it is: a proxy between the runner and the model provider that logs every request and response, with routing, keys, budgets and caching alongside.
When a factory picks it: the user wants per-run spend caps or one key vault for many runners, and logging comes with it; or the harness has no export of its own. It sees model calls only, so pair it with Type 2 or 3 for tool calls.
Starting points: LiteLLM Proxy (opens in a new tab) (open source, forwards logs to Langfuse or OTel), Portkey (opens in a new tab).
Verify: per-request pricing on SaaS, and whether the product is still actively developed (Helicone went into maintenance mode after an acquisition, as of 2026-10).
Check each candidate against the questions that decide this slot:
Where the answers live: the vendor’s pricing page (units, tiers, retention); the vendor’s integration doc for the specific harness; the harness’s own monitoring docs (what it exports, which flags are beta); the OpenTelemetry GenAI semantic conventions; the product’s repo for licence and release activity; the vendor’s changelog or blog for ownership changes.
What goes stale fastest: free-tier limits and prices; beta and preview flags (Claude Code trace export, Console platform push, vendor agent views); ownership and maintenance status; billing units; retention days; GenAI span names.