How do you stop one agent from having to know everything?
A cheap, fast classifier reads the incoming request first and routes it to one of several narrow, single-purpose agents — each holding only the tools, context, and instructions its job needs — instead of a single god-agent carrying every tool definition and every rule for every possible request. The router's only job is picking the right specialist; the specialist's only job is the task it was built for.
Also known as router-and-specialists, orchestrator-worker pattern, triage agent
What problem does this solve?
The obvious way to build an agent is to give it every tool it might ever need and a system prompt covering every case, then let the model figure out which parts apply this turn. That works until the tool count and instruction set both grow — past a certain point, more of the model's context budget goes to describing capabilities it won't use this turn than to reasoning about the one it will. It's also where tool-selection accuracy quietly drops, because two tools with overlapping-sounding descriptions become genuinely ambiguous to distinguish.
Reliability and instruction-following measurably degrade as context grows, even before the window fills. A god-agent handling billing, scheduling, and account changes has to hold all three domains' rules simultaneously, so a rule from one domain can bleed into a decision it was never meant to govern, and a change to one domain's instructions risks silently affecting the other two.
How does it work?
A small, cheap model — or a classifier that isn't an LLM at all — reads only the incoming request and decides which of several downstream specialist agents should handle it. The router carries no domain tools and no domain instructions; its entire prompt is the taxonomy of intents and the mapping to a specialist. Because it's classifying, not solving, it can run on a fast, inexpensive model and still hit high accuracy.
Each specialist is built for one job: a billing agent that only has billing tools and knows only billing policy, a scheduling agent that only has calendar tools. Context per specialist stays small because it was never diluted with instructions for the other domains, so instruction-following on its actual job improves rather than degrades from the god-agent baseline. Specialists can be versioned, tested, and rolled back independently — a bad prompt change to the scheduling agent can't touch billing at all, a testability property the single-agent version structurally can't have.
When the router is genuinely unsure, the correct move is to escalate to a general-purpose fallback agent or ask a clarifying question — never to guess and hand the request to the wrong specialist, which fails downstream in a way that's harder to diagnose than a router miss.
What are the moving parts?
- Classify first, solve nowhere
The router model receives only the raw request and outputs a single intent label plus a confidence score — it never attempts to answer the request itself.
- Map the label to exactly one specialist
A fixed lookup table, not the router's judgement at runtime, decides which specialist owns which intent, so routing is deterministic and auditable after the fact.
- Hand off a fresh, narrow context
The specialist starts from a short handoff summary and its own tool set, not the router's full transcript — it never inherits tools or instructions it doesn't need.
- Specialist executes within its own domain
Every tool call the specialist makes is scoped to its domain by construction, since it was never given tools outside it.
- Escalate below a confidence floor
A confidence score under a set threshold routes to a general fallback agent or a clarifying question, never to a best-guess specialist.
- Log the routing decision on every request
Store which intent was chosen, the confidence score, and which specialist handled it, so misrouting shows up as a measurable rate, not an anecdote.
When does it fit, and when doesn't it?
Use it when
- The agent's tool count has grown past what fits comfortably in one system prompt without diluting instruction-following
- Distinct domains have genuinely different tools, policies, or risk profiles (e.g. read-only lookups vs. account-mutating actions)
- You need to test, version, or roll back one domain's behavior without risking the others
- Different domains would benefit from different underlying models — a cheap model for FAQ-style intents, a stronger one for complex ones
Don't use it when
- You have one or two domains and a handful of tools — the router adds a hop and a failure point with nothing to divide
- Requests routinely span multiple domains in a single turn (e.g. 'reschedule and rebill'), which a hard classification boundary handles badly
- You don't yet have the traffic volume to build and maintain a labelled routing eval set — an unvalidated router is just an extra place to be wrong
- The cost of a wrong specialist handling a request safely, with a clean error, is low — a single well-scoped agent that occasionally does more than needed is simpler
What does it actually cost?
Each specialist's context stays small and focused, improving instruction-following on its own job
An added network hop and router latency on every single request, even the simple ones
Domains can be tested, versioned, and rolled back independently
A second system to maintain and eval — the router itself needs its own labelled test set and accuracy tracking
Blast radius of a bad prompt change is contained to one specialist
Cross-domain requests need an explicit multi-step or multi-agent handoff design, which is genuinely harder to build than a single capable agent
Cheaper models can handle the router and simple specialists, reserving spend for the domains that need it
A misrouted request is a distinct failure mode of its own that has to be measured and monitored, on top of each specialist's own error rate
How do you know it's working?
- Routing accuracy against a labelled eval set of intent examples
- Misroute rate — confirmed via user correction or specialist rejection — as a rate, not a raw count
- Router latency added per request (p50/p95)
- Confidence-score distribution, watched for drift toward the escalation floor
- Escalation rate to the fallback agent, and whether it rises over time
What failure modes does this prevent?
A Ship Audit checks which of these patterns your system actually needs, prioritized against what's most likely to break first.