Tool design

The Agent Tool Interface.

Six surfaces, 24 named design rules, and five ways a tool interface fails. The tools and the context an agent works through are the product surface, and most "the agent isn't reliable" complaints are interface problems, not model problems.

An agent doesn't experience your system the way a person does. It experiences a list of tool names, a schema per tool, whatever context got assembled ahead of it, and whatever comes back from the call it decided to make. Every one of those is a design decision someone made, deliberately or by default, and a system that looks unreliable is very often a system whose interface was never designed at all.

This isn't a case for a specific protocol or a specific framework. MCP standardizes how a tool call crosses a process boundary; it says nothing about how many tools you should expose, what to name them, or what a good response looks like. Those calls are still yours to make, on every surface below, whether the transport is MCP or a plain function call in your own codebase.

The six surfaces

Six surfaces, each blind to the next.

Every surface controls one part of the interface and is blind to the rest. That's exactly why getting one right doesn't get you the others.

Tool catalog & naming

Controls: Which tools exist in a session, and what they are named.

Blind to: Whether an individual tool's schema is well designed once it's picked.

Get it wrong: Near-identical tools crowd the list and the model guesses between them.

Tool contracts (the schema)

Controls: The shape of a single tool's inputs.

Blind to: Whether the result that comes back is any good.

Get it wrong: A loose schema lets the model invent a plausible-looking argument instead of supplying a real one.

Tool responses (what comes back)

Controls: What the model actually reads on the way back from a call.

Blind to: Whether the call was the right one to make in the first place.

Get it wrong: A response built for a human in a debugger buries or drops the one thing the model needed.

Context assembly & budget

Controls: What actually occupies the token budget at each step: tool defs, retrieved content, history, the task.

Blind to: Whether any individual piece placed in context is trustworthy.

Get it wrong: The task itself is the first thing truncated, because it was assembled last with no reserved share.

Permissions & blast radius

Controls: What a tool call is actually allowed to do once it executes.

Blind to: Whether the call needed to happen at all, or was named and scoped sensibly.

Get it wrong: A shared, broad credential means every bad call inherits the reach of the most dangerous tool in the session.

Transport & operations

Controls: How a tool call crosses a process or team boundary, and whether anyone can reconstruct what happened afterward.

Blind to: Whether the tool on the other side of that boundary is well designed on the inside.

Get it wrong: A protocol ships with no operations story behind it — nobody can say who called what, or trust the version they pinned.

The rule bank

24 rules, six surfaces.

Filter by applicability (read-only, writes, autonomous), then copy the visible list as a Markdown checklist.

01 Tool catalog & naming

Every tool definition sits in context on every single turn, whether or not that turn needs it — a session with forty tools pays for forty schemas before the model reasons about anything real. The catalog layer decides which tools exist at all, what they're called, and how many a session actually loads, and it's usually the first place a token budget quietly disappears. Get the count and the names right and most of tool-selection accuracy follows for free; get it wrong and no amount of context engineering downstream fixes a model choosing between two tools it can't tell apart.

Consolidate tools around workflows, not one tool per API endpoint.

Read-only

Why: A tool count that keeps growing widens the set of near-identical choices a model has to pick between on every turn — more tools raises the odds of a wrong pick, not the odds of a right one.
Build it: Group calls that share a workflow into one tool with a mode or action parameter (one `manage_ticket` tool with create/update/close, not three), and retire a tool outright once a broader one already covers its job.
Prove it: Track tool-choice accuracy on a scripted scenario set as the catalog grows (the trajectory layer on /evals/), and treat a drop as a signal to consolidate, not a signal to write a better prompt.

Namespace every tool name by the system that owns it.

Read-only

Why: Two tools named `search` from different systems are indistinguishable to a model reading a flat tool list — it has no way to know which `search` is Jira's and which is the internal wiki's.
Build it: Prefix every tool name with the system it belongs to (`jira_search`, `wiki_search`), so ownership is legible from the name alone, before the model ever reads the description.
Prove it: Grep the live tool list for a name collision or a missing prefix before every new integration ships, and add a lint check once the catalog is assembled from more than one source.

Give each tool exactly one job — never a read and a write behind the same name.

Writes

Why: A tool that reads under most calls and quietly writes under a specific parameter combination gets called with the read half's confidence and the write half's consequences.
Build it: Split any tool whose action depends on a parameter value into two tools, one read and one write, each with its own name, its own schema, and its own permission grant.
Prove it: Audit the catalog for a single tool name backing more than one permission scope, and treat a match as a design defect to fix, not a convenience worth keeping.

Cap the tool catalog per session and select the subset the task actually needs.

Read-only

Why: Every tool definition sits in the context window on every turn whether or not that turn needs it — a forty-tool catalog is a permanent tax the task pays regardless of which four tools it actually calls.
Build it: Cap the number of tools loaded per session, and select the subset a task needs at session start (via an intent router, or a fixed profile per task type) instead of loading the entire catalog by default.
Prove it: Measure the token share tool definitions occupy at session start, track it next to the task's own tokens, and alarm when it crosses a stated budget — see budget-the-window below.

Pattern: Intent router to specialists

02 Tool contracts (the schema)

A tool's schema is the only conversation the model has with the tool before it makes the tool call. No comments, no onboarding doc, just field names, types, and whichever constraints the schema author bothered to encode. Leave that loose and the model fills the gap the way it fills any gap: by generating something plausible-shaped, whether or not a real value exists behind it. Tool calling is only as reliable as this contract; 'the model hallucinated an argument' gets fixed here, with an enum or a validated pattern, not a better prompt.

Name every parameter for what it is, not a generic placeholder.

Read-only

Why: A parameter named `user` leaves open whether it wants an ID, an email, or a display name — the model fills the ambiguity with whichever it saw most in training, not whichever the tool actually expects.
Build it: Name every parameter for the specific thing it is (`user_id`, not `user`; `start_date_iso`, not `date`), and state the unit or format in the name itself wherever one exists.
Prove it: Read every parameter name cold, with no description attached, and confirm it's unambiguous on its own — if it needs the description to disambiguate, rename it instead of documenting around it.

Constrain known value sets to an enum in the schema, and validate it server-side.

Writes

Why: A free-text field for a known, bounded value set invites the model to generate a plausible-looking value instead of picking a real one — the same completion behavior that produces a hallucinated ID.
Build it: Constrain every field with a small, fixed value set to an enum in the schema, and validate server-side that whatever comes through actually belongs to it — never trust the schema alone to enforce itself.
Prove it: Track the invalid-argument rate on enum-constrained fields against free-text ones (the tool-call-argument-check on /evals/), and confirm the gap is large; if it isn't, the enum isn't constraining anything.

Pattern: Typed tool contract Failure mode: Tool argument hallucination

Mark a field required only when the tool genuinely can't act without it — and mean it.

Read-only

Why: A field marked optional that the tool actually needs just moves the failure downstream — the call succeeds at the schema layer and fails at execution, with a worse error and less context to explain why.
Build it: Mark `required` based on what the tool genuinely needs to act correctly, not on what's convenient to default — if a missing field always produces a degraded or wrong result, it's required.
Prove it: Sample recent execution failures and check how many trace back to a field that was optional in the schema but effectively mandatory in practice — each one is a schema fix, not a validation fix.

Write the tool description for a new hire who has never seen your API.

Read-only

Why: A tool description written for someone who already knows the underlying API assumes exactly the context the model doesn't have — units, formats, and relationships between resources have to be stated, not implied.
Build it: Write the description the way you'd brief a new hire on their first day: what the tool does, what a valid input looks like, what units and formats it expects, and how it relates to the tools around it.
Prove it: Hand the schema and description alone to someone unfamiliar with the system and ask them to predict a valid call; a description that requires guessing needs a rewrite.

03 Tool responses (what comes back)

What a tool sends back becomes the model's entire picture of what just happened. There's no side channel for 'this actually failed' unless the response format carries one. A raw database dump, a wall of UUIDs, an HTTP 200 wrapping an error string: none of these are wrong exactly, they're optimized for a human glancing at a debugger, not a model reasoning about the result next turn on a fixed token budget. That's what 'high-signal' actually buys on this layer: a model that notices a failure instead of one that reports an absence of data as a fact.

Return names and human-readable identifiers, not bare UUIDs, wherever one exists.

Read-only

Why: A response full of UUIDs gives the model a value it can pass to the next call but nothing it can reason about — it can't tell two records apart the way a human would by name.
Build it: Include a human-readable name or label alongside every identifier a response returns, so the next reasoning step has something legible to check its work against, not just an opaque token to forward.
Prove it: Read a sample of raw tool responses cold and check whether the meaningful content is legible without a lookup — if every row is an ID and nothing else, add a name field.

Return only what the next decision needs, not the whole record.

Read-only

Why: A full record dump forces the model to find the one relevant field itself, on every call, at the cost of every other field's tokens — and the noise makes the actual signal easier to miss, not just more expensive.
Build it: Return only the fields the next decision actually needs by default, with an explicit parameter to request more detail when a task genuinely calls for it — narrow by default, wide on request.
Prove it: Measure average response size in tokens against the share of fields a downstream step actually references (cost-per-completed-task on /evals/), and trim what nothing ever reads.

Paginate with a sane default limit, and tell the model exactly how to ask for more.

Read-only

Why: An unbounded result set either blows the context budget on one call or gets silently truncated with no signal that anything was cut — either way the model doesn't know it's looking at a partial answer.
Build it: Set a sane default page size on every list-returning tool, and when a response is truncated, say so explicitly and state exactly how to narrow the next call — a filter to add, a cursor to pass.
Prove it: Deliberately trigger a truncated response and confirm the model, reading only what came back, can construct a correctly narrowed follow-up call without being told how.

Make every error message a specific, actionable fix, never a stack trace.

Writes

Why: A stack trace or a generic 500 tells the model something failed and nothing about what to do next — it has no route to a fix, so it either repeats the identical call or gives up.
Build it: Return a specific, actionable error — 'no matching record for order 4471, check the order still exists' — instead of an implementation-level exception, and give write-scoped tools a distinct error path from read-scoped ones.
Prove it: Deliberately fail a call and confirm the returned error alone is enough for the model to correct its next attempt — if it just repeats the same call, the error didn't teach anything.

Failure mode: Silent tool failure

04 Context assembly & budget

Every tool definition, every retrieved document, and every prior turn competes for the same fixed context window, and nothing forces that competition to resolve in the task's favor. Left unmanaged, the stable stuff (system instructions, tool schemas) drifts toward the front, where a single reordered field busts prompt caching, while the volatile stuff (this turn's actual question) gets buried or evicted first when something has to go. Budgeting context is a decision made once, on purpose, not a truncation strategy improvised the day the window fills up.

Put static instructions and tool definitions first, and anything that changes per turn last.

Read-only

Why: A single volatile token near the front of a prompt (a timestamp, a reordered tool list, a per-request ID) busts prompt caching for the entire prefix behind it, even though the request looks basically unchanged to a human reading it.
Build it: Order every context assembly the same way on every call: static system instructions and tool definitions first, retrieved content next, whatever changes per turn last — and never reorder tools between calls in the same session.
Prove it: Check your actual cache-read share against provider usage data, not against how stable the prompt looks, and treat a low hit rate as an ordering bug to find.

Pattern: Stable-prefix prompt caching Failure mode: Prompt cache never hits

Write down the token budget per context slice, and alarm when one slice crosses it.

Read-only

Why: Without a stated allocation, tool definitions, retrieved content, and history all expand to fill whatever room is available, and whichever one grew fastest quietly starves the others — usually the task, since it's assembled last.
Build it: Write down a token allocation per slice (instructions, tool defs, retrieved content, history, task) before the system ships, and alert when any slice crosses its share instead of discovering it from a truncated task.
Prove it: Instrument actual token spend per slice on a sample of real sessions and compare it against the written budget — a slice that's silently 3x its allocation is the thing to fix before it causes an overflow.

Failure mode: Context overflow drops the task

Compact context on an explicit, stated rule, never as an emergency response to overflow.

Autonomous

Why: Compacting only once the window is already full means the eviction happens under pressure, with whatever's oldest or least structured cut first — usually exactly the state a long-running agent needed to keep.
Build it: Define the compaction rule in advance (summarize turns older than N steps, or once a slice crosses a stated share of budget) and run it on that schedule, not as a reaction to a budget already exceeded.
Prove it: Force a long run past the point compaction should trigger and confirm the agent's task state and constraints survive intact — if it forgets its own goal afterward, the rule cut the wrong thing.

Failure mode: Context overflow drops the task

Keep retrieved content and tool output structurally marked as data, never as instruction.

Writes

Why: A model reads one undifferentiated stream of tokens with no privileged channel marking 'this is an instruction' apart from 'this is retrieved text' — a document that reads like a command is functionally indistinguishable from one.
Build it: Mark retrieved content and tool output as data wherever the format allows it (a distinct field, a wrapping delimiter the system prompt explicitly names as untrusted), and never concatenate it directly into the instruction stream.
Prove it: Run a labelled set of injection attempts planted inside retrieved content and tool responses through the full tool-enabled path (the injection-red-team-set check on /evals/), and track resistance as its own number.

Failure mode: Injection via retrieved content

05 Permissions & blast radius

A tool call that can write, send, or delete is a different category of risk than one that can only read. Treating them the same, with one flat credential, one shared identity, and no distinction in the catalog, means every prompt injection, every hallucinated argument, and every over-eager retry inherits the full blast radius of the most dangerous tool in the session. This layer doesn't stop a bad call from being attempted; it decides how far a bad call can actually reach before something requires a human or a rollback.

Default every new tool to read-only, and make a write a separate tool with its own grant.

Writes

Why: A single set of credentials covering both read and write access means every read call carries the blast radius of the write it was never going to make — a hallucinated argument or a successful injection inherits permissions it didn't need.
Build it: Grant read access by default, and make any write a genuinely separate tool with its own, narrower credential — never one shared scope that happens to allow both.
Prove it: Audit the credential behind every tool and confirm a read-only tool's credential literally cannot perform a write at the provider level, not just that the tool's code chooses not to call it.

Put an idempotency key on every write so a retried call can't repeat the mutation.

Writes

Why: A write-scoped call that times out and gets retried, by the harness, by a bounded-loop budget, or by the model itself, either mutates twice or the retry becomes the actual source of runaway spend.
Build it: Generate a unique idempotency key per intended write and pass it through every retry of that same logical call, so the receiving system recognizes a repeat and no-ops it instead of executing it again.
Prove it: Deliberately retry a write call with the same key and confirm the second attempt is a no-op, then confirm a genuinely new call with a new key still goes through.

Failure mode: Retry-amplified spend

Require a human checkpoint before anything that can't be cleanly undone.

Autonomous

Why: An agent that can act autonomously has no backstop once a bad decision reaches an irreversible tool (a refund, a delete, a message sent externally). The call executes exactly as fast as a good one would.
Build it: Require an explicit human confirmation before any tool call that can't be cleanly undone, at minimum until the surrounding controls have an established track record — gated in the harness, not a suggestion in the prompt.
Prove it: Script an adversarial scenario that tries to make the agent issue a refund or send a message it shouldn't, and confirm the checkpoint actually fires — the tool-abuse-scenarios check on /evals/.

Pattern: Human checkpoint before irreversible actions

Flag every destructive or open-world tool explicitly, in the schema, not in a wiki page.

Writes

Why: A tool that deletes, sends, or reaches an open-world resource looks identical to a safe read in a flat tool list with no distinguishing marker, so nothing stops it being treated with the same casualness as a lookup.
Build it: Add an explicit annotation to every tool definition that's destructive, irreversible, or reaches outside a closed, known set of resources, so the harness and any reviewer can filter on it mechanically rather than reading every schema by hand.
Prove it: Grep the live catalog for every write-capable or open-world tool and confirm each one actually carries the annotation — an unmarked destructive tool is a defect to fix before the next session loads it.

06 Transport & operations

The moment a tool call crosses a process boundary (a team's API, a vendor's MCP server, a different product entirely), it stops being a function call and becomes a contract with an operations story: who's allowed to call it, what happens when its shape changes, and whether anyone can reconstruct what happened after the fact. MCP standardizes the wire format for that crossing. It does not standardize the governance around it, and most of what goes wrong at this layer is an org that shipped the protocol and skipped the operations.

Stay in-process until a tool genuinely crosses a team or product boundary.

Read-only

Why: Standing up a server, a transport layer, and an auth story for a tool that's only ever called from one process adds real operational surface for a boundary that doesn't exist yet.
Build it: Keep a tool in-process (a plain function call inside your own codebase) until it genuinely needs to cross a team or product boundary, and reach for MCP specifically at the point that boundary becomes real.
Prove it: Before standing up an MCP server for a new tool, name the specific other team or product that will call it — if the honest answer is 'nobody yet,' the boundary hasn't arrived.

Glossary: Model Context Protocol (MCP)

Version every tool contract, and give agents a real deprecation window before a breaking change.

Read-only

Why: A tool's schema that changes shape with no version marker breaks every agent that already pinned to the old shape, silently, the moment the new version deploys — there's no compiler to catch it.
Build it: Version every tool contract explicitly, keep the old version live through a stated deprecation window, and require agents to pin a version rather than always resolving to whatever is newest.
Prove it: Ship a deliberate breaking change behind a new version and confirm an agent still pinned to the old one keeps working through the entire deprecation window, not just at the moment of the change.

Glossary: Model Context Protocol (MCP)

Give every agent its own scoped, vaulted identity, never a shared service account.

Autonomous

Why: A shared service account behind every agent in a fleet makes it structurally impossible to answer 'which agent did this' after the fact, and means one agent's scope is every agent's scope.
Build it: Give every agent its own credential, scoped to only what that agent's tools actually need, vaulted rather than embedded in a prompt or a config file checked into source.
Prove it: Pick any tool call in a trace and confirm you can name the specific agent identity that made it, straight from the credential, not the shared account or the underlying human.

Log every tool call as structured data: tool, arguments, outcome, latency, tokens, cost.

Autonomous

Why: Without a call-level log, a regression in cost, latency, or reliability is a vague complaint about the agent getting slower, with no specific call, argument, or version to trace it back to.
Build it: Log tool, arguments, outcome, latency, tokens, and cost on every call, structured and queryable, not just the failures — the same trace-first discipline the handbook's operate stage applies to the rest of the system.
Prove it: Pick a random recent session and reconstruct its full tool-call trace, cost included, from the logs alone — if any call is missing or its cost can't be attributed, the logging has a gap.

Pattern: Trace-first observability

Five ways a tool interface fails

These aren't model problems.

Each one is a cause on this page's surfaces above, and a named symptom already documented elsewhere on this site.

Catalog creep

Looks like: Two or three tools in the same session do nearly the same thing under slightly different names, and the model picks whichever one it saw more often in training, not whichever one the task actually needed.

Guardrail: Audit the catalog for near-duplicates before adding a new tool, and merge or retire the older one instead of shipping a fourth way to do the same call.

Failure mode: Tool argument hallucination

Context crowd-out

Looks like: Tool definitions and retrieved results together eat most of the window before the model reaches the actual question, so the task itself is the first thing truncated when something has to give.

Guardrail: Budget context per slice (instructions, tool defs, retrieved content, task) and alarm when a slice crosses its share, instead of discovering the task got dropped after the fact.

Failure mode: Context overflow drops the task

Cheerful failure

Looks like: A tool call returns HTTP 200 with an empty or malformed body, and the model treats the absence of data as a fact worth reporting rather than a failure worth flagging.

Guardrail: Give every response a typed status field the model can branch on (ok, empty, error) so a failure has to be represented as one, not inferred from a body that happens to be blank.

Failure mode: Silent tool failure

Retry storm

Looks like: A write-scoped tool call times out, gets retried automatically, and the mutation it was supposed to make either happens twice or the retry itself becomes the thing burning budget.

Guardrail: Put an idempotency key on every write call so a retried request is provably a no-op, not a second mutation, and cap retries explicitly instead of leaving the ceiling unbounded.

Failure mode: Retry-amplified spend

Instructions in the payload

Looks like: A tool's response includes retrieved text that reads like an instruction. The model, with no structural way to tell data from command, follows it.

Guardrail: Treat everything a tool returns as data, never as instruction, and scope every tool to least privilege so a followed instruction still has nowhere dangerous to go.

Failure mode: Injection via retrieved content

What this is built from

Verifiable, not claimed.

No invented benchmark numbers, no client names. Just what's already documented on this site.

Typed tool contract and bounded agent loop are patterns this site already documents in full — this page is what ties their design rules into one coherent interface instead of two standalone techniques. Pattern: Typed tool contract
Tool argument hallucination, context overflow, silent tool failure, retry-amplified spend, and injection via retrieved content are named failure modes on this site, not hypotheticals — each is the reason a specific rule above exists. Failure mode: Silent tool failure
CodeHerder runs its own tool catalog and context budget across an entire fleet of coding agents — the catalog, context, and permissions surfaces above, applied at fleet scale instead of to one integration. CodeHerder
CostMon is being built to answer the transport layer's per-call cost line (log every call, above) as a self-serve number instead of a monthly reconciliation project. CostMon (coming soon)
The Model Context Protocol glossary entry is the standard this page's transport surface is built around adopting deliberately, not by default. Glossary: Model Context Protocol (MCP)

Not sure which surface is causing it

A Ship Audit checks the tool interface an agent actually runs against (catalog, contract, permissions, and the rest) and tells you which surface is the live risk, not just which one is theoretically incomplete.

Book a Ship Audit → See the Design stage →

Questions

Before you write the schema.

What teams ask us before they scope a tool catalog.

01 How many tools is actually too many?

There's no fixed number — the failure mode is near-identical tools crowding the model's choice, not a raw count. Audit for overlap before adding a new one, and cap what loads per session to what the task needs; see catalog-under-a-budget above.

02 Do we need MCP, or is in-process enough?

Stay in-process until a tool genuinely crosses a team or product boundary. MCP earns its cost (a server, a transport layer, an auth story) once another team or product actually needs to call the same tool; standing it up earlier is operational surface with no boundary to justify it.

03 What's the single highest-leverage rule to start with?

Read-only by default, paired with a typed status field on every response. Together they close off the two failure modes that do the most damage early: an over-scoped credential, and a silent failure the model reports as fact.

04 How does this relate to the Eval Stack?

The Eval Stack tells you whether the system works; this page is the interface the agent works through to get there. A contract check on /evals/ is scored against a tool contract built to the rules above — the two are complementary, not overlapping.

05 Isn't this just prompt engineering?

No. Prompt engineering shapes what you ask the model; this is the surface the model acts through once it decides to do something. A well-designed tool catalog survives a mediocre prompt; a great prompt on top of a badly designed tool interface still fails on the tool call.

06 What's context engineering, and how is it different from prompt engineering?

Context engineering is deciding what occupies the token window at every step (tool defs, retrieved content, history, the task itself) and in what order, not just what to say in the system prompt. Prompt engineering is one input to it, not the whole discipline; see the context surface above.