CustomLabs
Evals & observability

How do you find out why one specific run went wrong?

Intent

One trace ID follows a single request across every hop it takes — retrieval, every model call, every tool call — logged with enough detail to reconstruct exactly what happened after the fact. A sampled slice of traces, weighted toward low-confidence or error-flagged runs, lands in a queue a human actually reviews on a schedule, not only when a complaint forces someone to go looking.

Also known as distributed tracing for agents, end-to-end trace ID, sampled human review queue

Problem

What problem does this solve?

Most AI systems accumulate logs — a retrieval log here, a model-call log there, a tool-call log somewhere else — with no shared identifier tying them to the same request. Reconstructing what happened on one bad run means manually correlating timestamps across systems that weren't built to be correlated, which is slow enough that most teams only do it once a user complaint forces the question.

Aggregate metrics — average latency, overall error rate — can look healthy while a specific, real failure sits unexamined in the tail, because nobody has a habit of looking at individual traces until something breaks loudly enough to demand it. By the time someone goes looking, the context needed to diagnose it, what exactly the model saw at each step, may be gone or hard to reassemble.

Mechanics

How does it work?

A single trace ID is generated at the start of each request and threaded through every subsequent hop — retrieval calls, every model invocation, every tool call — so the full sequence for one request can be pulled with one query, not reconstructed from scattered logs after the fact.

Each span in the trace records enough to actually diagnose a failure: the exact prompt sent, the exact response received, which tool was called with which arguments, and what came back, not just a duration and a status code. This is the raw material every other observability and eval practice depends on; without it, a "we can't reproduce it" conversation is the ceiling of what's diagnosable.

Rather than reviewing every trace, which is infeasible at volume, or none, which is where most teams default to, a sampling policy pulls a slice into a queue a human actually reviews on a schedule, weighted toward traces the system itself flagged as low-confidence, erroring, or unusually long, so the sample catches the cases most worth a human's attention, not a uniform random cross-section.

Shape

What are the moving parts?

  1. Generate one trace ID per request

    Assign it at the entry point, before the first hop, so nothing downstream is uncorrelated.

  2. Thread the ID through every downstream hop

    Carry it through every retrieval call, model call, and tool call for that request.

  3. Log the actual content per span

    Record the real prompt, response, and tool arguments, not just timing and status.

  4. Weight sampling toward flagged runs

    Favor low-confidence, error-flagged, or unusually long runs over a uniform random sample.

  5. Route the sample into a real human review queue

    Send it to a queue with an owner and a schedule, not an unread archive.

  6. Feed findings back into evals or prompts

    Close the loop by turning what review surfaces into new eval cases or prompt fixes.

Fit

When does it fit, and when doesn't it?

Use it when

  • A support or engineering escalation currently requires manually correlating logs across two or more systems to explain one bad run
  • The system has more than a couple of hops (retrieval + model + tool calls) where a failure could originate in any of them
  • You want a repeatable way to catch drift or degradation before it's reported, not only after
  • You are already building or maintaining a golden-set eval and need a source of real, hard cases to feed it

Don't use it when

  • The system is a single model call with no tools or retrieval — there is no multi-hop sequence that needs correlating
  • You have no plan or team capacity to actually staff the human review queue — collecting traces nobody reviews is storage, not observability
  • Traffic volume is low enough that reviewing every single run by hand is already realistic — sampling infrastructure would be solving a problem you do not have yet
Trade-offs

What does it actually cost?

Gain

A bad run becomes explainable from one query instead of a cross-system log correlation exercise

Cost

Storing full prompts, responses, and tool payloads per span raises storage volume and cost, especially at scale

Gain

A weighted sample surfaces the runs most worth human attention, not a random slice

Cost

The review queue needs a real, staffed owner and cadence, or the sampled traces just accumulate unread

Gain

Traces become the raw material for eval-set expansion and root-cause diagnosis alike

Cost

Full prompt/response logging raises data-handling and retention questions that need an explicit policy, not an afterthought

Gain

Drift and degradation become visible before a complaint forces the investigation

Cost

Threading one trace ID through every hop is real integration work across every service the request touches, not a config flag

Signals

How do you know it's working?

  • Time-to-diagnose for a reported bad run, before and after adopting trace-first logging
  • Sampling coverage — what share of low-confidence/error-flagged runs actually reach a human reviewer
  • Review queue backlog and age, so a growing unreviewed backlog is caught early
  • Count of review findings that fed back into the eval set or a prompt change
Prevents

What failure modes does this prevent?

Not sure this is the right pattern?

A Ship Audit checks which of these patterns your system actually needs, prioritized against what's most likely to break first.

Book a Ship Audit

navigate select esc close