How do you stop one agent from having to know everything?
A cheap, fast classifier reads each request and routes it to one narrow specialist agent, instead of one god-agent that carries every tool. Each specialist holds only the tools, context, and instructions its job needs. The router's only job is picking the right specialist, and the specialist's only job is the task it was built for.
Also known as router-and-specialists, orchestrator-worker pattern, triage agent
What problem does this solve?#
The obvious way to build an agent is to give it every tool it might ever need, plus a system prompt covering every case. The model then has to figure out which parts apply this turn. That works until the tool count and instruction set both grow. Past a certain point, more of the model's context budget goes to describing capabilities it won't use this turn than to reasoning about the one it will. Tool-selection accuracy quietly drops too, because two tools with overlapping-sounding descriptions become genuinely ambiguous to distinguish.
Reliability and instruction-following measurably degrade as context grows, even before the window fills. A god-agent handling billing, scheduling, and account changes has to hold all three domains' rules at once. A rule from one domain can bleed into a decision it was never meant to govern. A change to one domain's instructions risks silently affecting the other two.
How does it work?#
A small, cheap model, or a classifier that isn't an LLM at all, reads only the incoming request. It decides which of several downstream specialist agents should handle it. The router carries no domain tools and no domain instructions. Its entire prompt is the taxonomy of intents and the mapping to a specialist. Because it's classifying, not solving, it can run on a fast, inexpensive model and still hit high accuracy.
Each specialist is built for one job: a billing agent that only has billing tools and knows only billing policy, a scheduling agent that only has calendar tools. Context per specialist stays small, because it was never diluted with instructions for the other domains. Instruction-following on its actual job improves rather than degrades from the god-agent baseline. Specialists can be versioned, tested, and rolled back independently. A bad prompt change to the scheduling agent can't touch billing at all, a testability property the single-agent version structurally can't have.
When the router is genuinely unsure, the correct move is to escalate to a general-purpose fallback agent or ask a clarifying question. It should never guess and hand the request to the wrong specialist. A wrong handoff fails downstream in a way that's harder to diagnose than a router miss.
What are the moving parts?#
- Classify first, solve nowhere
The router model receives only the raw request. It outputs a single intent label plus a confidence score, and never attempts to answer the request itself.
- Map the label to exactly one specialist
A fixed lookup table, not the router's judgement at runtime, decides which specialist owns which intent, so routing is deterministic and auditable after the fact.
- Hand off a fresh, narrow context
The specialist starts from a short handoff summary and its own tool set, not the router's full transcript. It never inherits tools or instructions it doesn't need.
- Specialist executes within its own domain
Every tool call the specialist makes is scoped to its domain by construction, since it was never given tools outside it.
- Escalate below a confidence floor
A confidence score under a set threshold routes to a general fallback agent or a clarifying question, never to a best-guess specialist.
- Log the routing decision on every request
Store which intent was chosen, the confidence score, and which specialist handled it, so misrouting shows up as a measurable rate, not an anecdote.
When does it fit, and when doesn't it?#
Use it when
- The agent's tool count has grown past what fits comfortably in one system prompt without diluting instruction-following
- Distinct domains have genuinely different tools, policies, or risk profiles (e.g. read-only lookups vs. account-mutating actions)
- You need to test, version, or roll back one domain's behavior without risking the others
- Different domains would benefit from different underlying models: a cheap model for FAQ-style intents, a stronger one for complex ones
Don't use it when
- You have one or two domains and a handful of tools. The router just adds a hop and a failure point with nothing to divide
- Requests routinely span multiple domains in a single turn (e.g. 'reschedule and rebill'), which a hard classification boundary handles badly
- You don't yet have the traffic volume to build and maintain a labelled routing eval set. An unvalidated router is just an extra place to be wrong
- The cost of a wrong specialist handling a request safely, with a clean error, is low. A single well-scoped agent that occasionally does more than needed is simpler
What does it actually cost?#
Each specialist's context stays small and focused, improving instruction-following on its own job
An added network hop and router latency on every single request, even the simple ones
Domains can be tested, versioned, and rolled back independently
A second system to maintain and eval: the router itself needs its own labelled test set and accuracy tracking
Blast radius of a bad prompt change is contained to one specialist
Cross-domain requests need an explicit multi-step or multi-agent handoff design, which is genuinely harder to build than a single capable agent
Cheaper models can handle the router and simple specialists, reserving spend for the domains that need it
A misrouted request is a distinct failure mode of its own that has to be measured and monitored, on top of each specialist's own error rate
How do you know it's working?#
- Routing accuracy against a labelled eval set of intent examples
- Misroute rate, confirmed via user correction or specialist rejection, as a rate rather than a raw count
- Router latency added per request (p50/p95)
- Confidence-score distribution, watched for drift toward the escalation floor
- Escalation rate to the fallback agent, and whether it rises over time
What failure modes does this prevent?#
Sources
- Model Context Protocol - MCP Specification: Tools (2026-07-28)
The specification text our tool-design and catalogue rules are checked against. Retrieved 2026-08-24.
- arXiv - Lost in the Middle: How Language Models Use Long Contexts
The paper measuring how a fact's position in a context window affects recall. Retrieved 2026-08-24.
A Ship Audit checks which of these patterns your system actually needs, prioritized against what's most likely to break first.