Agentic workflow runner for a back-office queue
An agent that works a queue of cases end to end, inside a bounded loop with a human checkpoint on anything it cannot cleanly undo.
“We have a queue our team processes by hand, and we want an agent to work through it without babysitting every case.”
What does the system look like?#
How does a request move through it?#
- Intake and classify
A case enters the queue and a cheap router classifies which specialist agent should own it, carrying no domain tools of its own.
- Hand off to a bounded specialist
The specialist starts from a short handoff summary and only the tools its domain needs, under an explicit step, token, and wall-clock budget.
- Work the case through typed tools
Every tool argument is validated against a strict schema before it executes; a value the agent can't source from a prior lookup never reaches a mutating call.
- Checkpoint before anything irreversible
A refund, a delete, an external message: anything that cannot be cleanly undone waits for an explicit human confirmation, no matter how confident the agent looked getting there.
- Terminate into a named state
Every run ends as exactly one of succeeded, failed-with-reason, budget-exceeded, or escalated, forced by the harness once a ceiling or a no-progress threshold is hit.
- Trace and feed the eval set
Every case writes a full trace and its terminal state, succeeded or not, so a stalled run is a queryable rate and an escalation is a new case in the next eval run.
What are the pieces, and what breaks without each one?#
Intent router
Classify each incoming case to exactly one specialist agent using a fixed lookup table, escalating below a confidence floor instead of guessing.
Breaks without it: A single agent tries to hold every domain's tools and rules at once, and instruction-following degrades on all of them together.
Bounded specialist agent
Work one case at a time under an explicit step, token, and wall-clock budget, with a forced terminal state when the budget runs out.
Breaks without it: A stuck agent keeps retrying a failed approach because the failed attempt sits in context looking exactly like new information to reason from.
Typed tool layer
Validate every tool call against a strict schema server-side, and require mutating arguments to trace back to a prior read result.
Breaks without it: The agent can call a backend system with a plausible-looking but invented ID, and nothing catches it before it executes.
Human checkpoint queue
Gate every irreversible action behind an explicit approval, staffed by a named owner with a real SLA.
Breaks without it: A wrong or successfully injected instruction reaches a refund, a delete, or an outbound message with nothing standing between the decision and its consequence.
Terminal-state logger
Record exactly one terminal state per run, including every run that succeeded.
Breaks without it: Nobody can answer "how often does this actually finish" without reading transcripts by hand.
Trace store
Capture the full prompt, tool call, and result sequence for every case under one trace ID.
Breaks without it: A stalled or wrong case becomes a "we can't reproduce it" conversation instead of a five-minute trace lookup.
Where does this need a decision, not a default?#
| Decision | Default choice | Why |
|---|---|---|
| Orchestration | A durable workflow engine (queue-backed, resumable) over a hand-rolled loop in application code | A case that spans a checkpoint approval, sometimes hours later, needs to survive a process restart; a workflow engine gives you that for free, a custom loop does not. |
| Agent harness | Adopt an existing agent framework for tool calling and budgets rather than build one from scratch | Step budgets, termination contracts, and tool-call validation are solved problems; the actual differentiation is in the tool set and the checkpoint policy, not the loop mechanics. |
| Human review surface | Route checkpoints into whatever tool your team already triages work in, not a new bespoke approval UI | A checkpoint queue with no real owner and no habit of checking it just becomes a silent drop; meeting reviewers where they already work is what keeps the SLA real. |
| Queue system | Extend the existing ticketing or case-management system the team already uses, rather than stand up a parallel one | A second source of truth for what's in the queue is exactly the kind of drift this architecture is trying to eliminate. |
What actually drives the bill?#
- Tool calls chained per case (the calls-to-completions ratio, not calls alone)
- Retries on transient tool failures inside the bounded loop
- Human review minutes per case that reaches a checkpoint
Dominant cost Chained tool calls per completed case once the agent is handling real volume, because a multi-step case pays for several calls, not one, and a stalling run pays for many more before its budget forces a stop.
The lever A hard step and token ceiling per run caps the worst case, and routing simple classification steps to a cheap model keeps the router itself nearly free.
Model your own numbers with the AI Cost Calculator →How do you know it actually works?#
Terminal-state rate across success, failure, and escalation
The blunt signal for whether the fleet is actually finishing cases or quietly looping.
Tool-choice accuracy against scripted scenarios
Checks the agent reaches an answer through the right tool sequence, not a lucky detour.
Tool-abuse scenarios against every write-scoped tool
Confirms the checkpoint actually fires, or the call is rejected outright, under adversarial pressure.
Cost per completed case, not per call
Surfaces a retry loop hiding inside a good-looking per-call average.
What will your reviewers ask about this?#
Blast-radius mapping for every reachable tool
A written list of what a compromised or wrong decision could actually do downstream, scoped per specialist.
A written list of which actions run unattended and which require a human, with every irreversible one on the human side by default.
Enforced step, token, and wall-clock budget
A harness-level ceiling, not a promise the model will decide when to stop.
Which patterns and failure modes tie in?#
How long does a first version actually take?#
6-10 weeks to a pilot handling one queue end to end with a checkpoint on every irreversible step; longer once more than one specialist or backend system enters the loop.
When is this the wrong shape?#
- The steps needed to resolve a case are fully knowable in advance. A fixed pipeline is cheaper to build and far easier to eval than an agent for a task that never actually branches.
- Nobody is willing to own the approval queue a human checkpoint depends on. An unstaffed checkpoint is a silent drop, not a safety control.
- The backend systems the agent would need to call have no API, only a screen a person already has to click through. Automating the screen is a different, usually worse, project than this one.
Sources
- NIST - AI Risk Management Framework (AI RMF 1.0)
The risk-management functions our governance and adoption controls map onto. Retrieved 2026-08-24.
- Google - Error Budget Policy
The error-budget policy pattern our release gate borrows. Retrieved 2026-08-24.
A Ship Audit checks which of these an existing or planned system actually needs, against what is most likely to break first.