Evaluation
Observability
Observability captures traces of every prompt, retrieval, tool call, and response for debugging.
It tracks quality, latency, and cost over time, beyond whether the request simply returned a 200.
Without it, a regression surfaces as a vague complaint that "the AI got worse," with no clear cause.
It is the operational counterpart to an eval suite: evals catch regressions before ship, this catches them after.
← Back to the full glossaryRelated terms
Agent (Agentic AI) An agent is an LLM given a loop, memory, and tools it can call. LLM-as-Judge LLM-as-judge uses another LLM to score a model's outputs against a rubric, at scale. Output Drift Output drift is a gradual change in a live system's output, with no release behind it. Shadow Deployment A shadow deployment runs a candidate release against real production traffic, offline, before release.
More in Evaluation
Eval Suite (Evals) An eval suite is a versioned set of test cases that measures whether outputs are good. Hallucination A hallucination is a confident, fluent output that is factually wrong or unsupported. Prompt Injection Prompt injection is untrusted input crafted to override a model's system prompt or task.