Agentic workflow runner for a back-office queue
An agent that works a queue of cases end to end. It runs in a bounded loop. A human checkpoint covers any action it cannot cleanly undo.
“We have a queue our team processes by hand, and we want an agent to work through it without babysitting every case.”
What does the system look like?#
How does a request move through it?#
- Intake and classify
A case enters the queue. A cheap router picks the specialist agent that owns it. The router has no domain tools of its own.
- Hand off to a bounded specialist
The specialist starts from a short handoff summary. It gets only the tools its domain needs. An explicit step, token, and wall-clock budget applies.
- Work the case through typed tools
Every tool argument is validated against a strict schema before it runs. A value that did not come from a prior lookup never reaches a mutating call.
- Checkpoint before anything irreversible
A refund, a delete, or an external message cannot be cleanly undone. Each one waits for explicit human confirmation. The confidence of the agent does not change this.
- Terminate into a named state
Every run ends as one of four states: succeeded, failed-with-reason, budget-exceeded, or escalated. The harness forces the end when a ceiling or a no-progress threshold is hit.
- Trace and feed the eval set
Every case writes a full trace and its terminal state. A stalled run becomes a rate you can query. An escalation becomes a new case in the next eval run.
What are the pieces, and what breaks without each one?#
Intent router
Classify each incoming case to one specialist agent with a fixed lookup table. Escalate below a confidence floor and do not guess.
Breaks without it: One agent tries to hold the tools and rules of every domain at once. Instruction-following gets worse in all of them.
Bounded specialist agent
Work one case at a time under an explicit step, token, and wall-clock budget. Force a terminal state when the budget runs out.
Breaks without it: A stuck agent keeps retrying a failed approach. The failed attempt sits in context and looks like new information.
Typed tool layer
Validate every tool call against a strict schema on the server. Require each mutating argument to trace back to a prior read result.
Breaks without it: The agent can call a backend system with an invented ID that looks plausible. Nothing catches it before it runs.
Human checkpoint queue
Gate every irreversible action behind an explicit approval. A named owner staffs the queue and works to a real SLA.
Breaks without it: A wrong or injected instruction can reach a refund, a delete, or an outbound message. Nothing stands between the decision and its consequence.
Terminal-state logger
Record one terminal state per run. Include every run that succeeded.
Breaks without it: Nobody can answer "how often does this finish" without reading transcripts by hand.
Trace store
Capture the full sequence of prompts, tool calls, and results for every case under one trace ID.
Breaks without it: A stalled or wrong case ends in a "we cannot reproduce it" conversation. With a trace, the lookup takes five minutes.
Which choices need a decision?#
| Decision | Default choice | Why |
|---|---|---|
| Orchestration | A durable workflow engine (queue-backed and resumable) in place of a hand-rolled loop in application code | A case can wait hours for a checkpoint approval. It must survive a process restart. A workflow engine gives you that for free. A custom loop does not. |
| Agent harness | Adopt an existing agent framework for tool calling and budgets. Do not build one from scratch | Step budgets, termination contracts, and tool-call validation are solved problems. Your edge is in the tool set and the checkpoint policy. The loop mechanics add little. |
| Human review surface | Route checkpoints into the tool your team already uses to triage work. Skip the new bespoke approval UI | A checkpoint queue with no owner and no checking habit becomes a silent drop. Meet reviewers where they already work. That keeps the SLA real. |
| Queue system | Extend the ticketing or case-management system the team already uses. Do not stand up a parallel one | A second source of truth for the queue is the kind of drift this architecture aims to remove. |
What drives the bill?#
- Tool calls chained per case, measured as the calls-to-completions ratio
- Retries on transient tool failures inside the bounded loop
- Human review minutes per case that reaches a checkpoint
Dominant cost Chained tool calls per completed case, once the agent handles real volume. A multi-step case pays for several calls. A stalling run pays for many more before its budget forces a stop.
The lever A hard step and token ceiling per run caps the worst case. Routing simple classification steps to a cheap model keeps the router nearly free.
Model your own numbers with the AI Cost Calculator →How do you know it works?#
Terminal-state rate across success, failure, and escalation
The blunt signal for whether the fleet finishes cases or loops.
Tool-choice accuracy against scripted scenarios
Checks that the agent reaches an answer through the right tool sequence and avoids a lucky detour.
Tool-abuse scenarios against every write-scoped tool
Confirms that the checkpoint fires or the call is rejected under adversarial pressure.
Shows a retry loop that hides inside a good-looking per-call average.
What will your reviewers ask about this?#
Blast-radius mapping for every reachable tool
A written list of what a compromised or wrong decision could do downstream. Scope the list per specialist.
A written list of which actions run unattended and which need a human. Every irreversible action needs a human by default.
Enforced step, token, and wall-clock budget
A ceiling set in the harness. The model does not decide when to stop.
Which patterns and failure modes tie in?#
How long does a first version take?#
6-10 weeks to a pilot that handles one queue end to end with a checkpoint on every irreversible step. Plan for longer once more than one specialist or backend system enters the loop.
When is this the wrong shape?#
- You know every step needed to resolve a case in advance. A fixed pipeline costs less to build and is far easier to eval than an agent. The task never branches.
- Nobody is willing to own the approval queue that a human checkpoint needs. An unstaffed checkpoint is a silent drop. It gives no safety.
- The backend systems the agent must call have no API. A person clicks through a screen. Automating that screen is a different project, and it is usually worse.
Sources
- NIST - AI Risk Management Framework (AI RMF 1.0)
The risk-management functions our governance and adoption controls map onto. Retrieved 2026-08-24.
- Google - Error Budget Policy
The error-budget policy pattern our release gate borrows. Retrieved 2026-08-24.
A Ship Audit checks which of these an existing or planned system needs. It ranks them by what is most likely to break first.