How do you stop one agent from having to know everything?
A fast, cheap classifier routes each request to one narrow specialist agent. Each specialist holds only the tools and instructions its job needs. The router picks the specialist. The specialist does the task.
Also known as router-and-specialists, orchestrator-worker pattern, triage agent
What problem does this solve?#
The obvious way to build an agent gives it every tool it might need and a system prompt that covers every case. The model must then work out which parts apply to this turn. That works until the tool count and the instruction set grow. Past a certain point, the model spends more of its context budget on capabilities it will not use than on the one it will use. Tool-selection accuracy also drops, because two tools with similar descriptions are hard to tell apart.
Reliability and instruction-following degrade as context grows, even before the window fills. A god-agent that handles several domains must hold all their rules at once. A rule from one domain can bleed into a decision it never governed. A change to one domain's instructions can affect the others.
How does it work?#
A small, cheap model reads only the incoming request. A classifier that is not an LLM can do the same job. The router decides which downstream specialist agent handles the request. The router carries no domain tools and no domain instructions. Its prompt holds the taxonomy of intents and the mapping to specialists. The router only classifies, so a fast, inexpensive model can reach high accuracy.
Each specialist does one job. A billing agent has only billing tools and knows only billing policy. A scheduling agent has only calendar tools. Context per specialist stays small, because no instructions for other domains dilute it. Instruction-following on its own job improves over the god-agent baseline. Teams can version and test specialists independently. They can also roll each one back alone. A bad prompt change to the scheduling agent cannot touch billing. The single-agent version cannot offer this testability.
When the router is unsure, it escalates to a general-purpose fallback agent or asks a clarifying question. It never guesses and hands the request to the wrong specialist. A wrong handoff fails downstream in a way that is harder to diagnose than a router miss.
What are the moving parts?#
- Classify first, solve nowhere
The router model receives only the raw request. It outputs one intent label and a confidence score. It never tries to answer the request.
- Map the label to exactly one specialist
A fixed lookup table decides which specialist owns each intent. Routing is therefore deterministic and auditable after the fact.
- Hand off a fresh, narrow context
The specialist starts from a short handoff summary and its own tool set. It inherits no tools or instructions that it does not need.
- Specialist executes within its own domain
Every tool call the specialist makes stays in its domain, because the specialist has no tools outside it.
- Escalate below a confidence floor
A confidence score under a set threshold routes to a general fallback agent or a clarifying question. No request goes to a best-guess specialist.
- Log the routing decision on every request
Store the chosen intent and its confidence score. Also store the specialist that handled the request. Misrouting then shows up as a measurable rate.
When does it fit, and when doesn't it?#
Use it when
- The agent's tool count has outgrown one system prompt, and instruction-following suffers.
- Distinct domains have different tools or risk profiles, such as read-only lookups and account-changing actions.
- You need to test or roll back one domain without risking the others.
- Different domains suit different models: a cheap model for FAQ-style intents and a stronger one for complex intents.
Don't use it when
- You have one or two domains and a handful of tools. The router adds a hop and a failure point, and nothing needs dividing.
- Requests often span several domains in one turn, such as 'reschedule and rebill'. A hard classification boundary handles these badly.
- You lack the traffic to build and maintain a labelled routing eval set. An unvalidated router is one more place to be wrong.
- The cost of a wrong specialist handling a request is low, because it fails safely with a clean error. A single well-scoped agent that sometimes does extra work is simpler.
What does it cost?#
Each specialist's context stays small and focused, which improves instruction-following on its own job.
The router adds a network hop and latency to every request, even simple ones.
Teams can test and roll back each domain independently.
The team maintains a second system. The router needs its own labelled test set and accuracy tracking.
A bad prompt change affects only one specialist.
Cross-domain requests need an explicit multi-step or multi-agent handoff design. That design is harder to build than a single capable agent.
Cheaper models can run the router and the simple specialists. The team saves spend for the domains that need it.
A misrouted request is a distinct failure mode. The team must measure and monitor it on top of each specialist's own error rate.
How do you know it's working?#
- Routing accuracy against a labelled eval set of intent examples
- Misroute rate as a share of requests, confirmed by user correction or specialist rejection
- Router latency added per request (p50/p95)
- Confidence-score distribution, watched for drift toward the escalation floor
- Escalation rate to the fallback agent, and its trend over time
What failure modes does this prevent?#
Sources
- Model Context Protocol - MCP Specification: Tools (2026-07-28)
The specification text our tool-design and catalogue rules are checked against. Retrieved 2026-08-24.
- arXiv - Lost in the Middle: How Language Models Use Long Contexts
The paper measuring how a fact's position in a context window affects recall. Retrieved 2026-08-24.
A Ship Audit checks which of these patterns your system needs. It ranks them by what is most likely to break first.