Observability

Observability, in an LLM context, means capturing traces of every prompt, retrieval, tool call, and response so a team can debug why a specific output happened and track quality, latency, and cost over time — not just whether the request returned a 200. Without it, a regression after a prompt or model change surfaces as a vague complaint that "the AI got worse," with no way to pinpoint which step changed. It's the operational counterpart to an eval suite: evals catch regressions before ship, observability catches them after.

← Back to the full glossary