Tool catalog & naming
Controls: Which tools exist in a session, and what they are named.
Blind to: Whether the schema of a chosen tool is well designed.
Get it wrong: Near-identical tools crowd the list, and the model guesses between them.
Tool design
The Agent Tool Interface.
Six surfaces, 24 named design rules, and five ways a tool interface fails. The tools and the context an agent works through are the product surface. Most complaints that an agent is unreliable trace back to the interface.
An agent sees your system differently from a person. It sees a list of tool names and a schema for each tool. It sees the context assembled ahead of it and the result of each call. A person made every one of those design decisions, on purpose or by default. A system that looks unreliable often has an interface that nobody designed.
These rules apply to any protocol and any framework. MCPMCP is an open standard for connecting LLM applications to tools and data sources. standardizes how a tool call crosses a process boundary. It says nothing about how many tools to expose, what to name them, or what a good response looks like. You make those decisions on every surface below. This is true for MCP and for a plain function call in your own codebase.
The six surfaces
Six surfaces, six blind spots.#
Each surface controls one part of the interface and cannot see the rest. A correct surface does not fix the others.
Tool contracts (the schema)
Controls: The shape of the inputs to one tool.
Blind to: Whether the result that comes back is any good.
Get it wrong: A loose schema lets the model invent a plausible argument instead of supplying a real one.
Tool responses (what comes back)
Controls: What the model reads when a call returns.
Blind to: Whether the call was the right one to make.
Get it wrong: A response built for a human in a debugger buries or drops the one thing the model needed.
Context assembly & budget
Controls: What occupies the token budget at each step: tool definitions, retrieved content, history, and the task.
Blind to: Whether any piece placed in context is trustworthy.
Get it wrong: The task is the first thing truncated, because it was assembled last with no reserved share.
Permissions & blast radius
Controls: What a tool call may do once it executes.
Blind to: Whether the call needed to happen, or whether its name and scope make sense.
Get it wrong: A shared, broad credential gives every bad call the reach of the most dangerous tool in the session.
Transport & operations
Controls: How a tool call crosses a process or team boundary, and whether anyone can reconstruct what happened afterward.
Blind to: Whether the tool on the other side of that boundary is well designed inside.
Get it wrong: A protocol ships with no operations story. Nobody can say who called what, or trust the version they pinned.
The rule bank
24 rules, six surfaces.#
Filter by applicability (read-only, writes, autonomous), then copy the visible list as a Markdown checklist.
Showing all 24 rules
01 Tool catalog & naming
Every tool definition sits in context on every turn. The model reads it whether or not that turn needs it. A session with forty tools pays for forty schemas before the model reasons about anything. The catalog layer decides which tools exist and what they are called. It also decides how many a session loads. This is the first place a token budget disappears. Get the count and the names right, and tool selection improves with little extra effort. Get them wrong, and no later context engineeringContext engineering decides what occupies a model's context window at every step, and in what order. helps a model that cannot tell two tools apart.
- Why
- A growing tool count widens the set of near-identical choices the model faces on every turn. More tools raise the odds of a wrong pick.
- Build it
- Group calls that share a workflow into one tool with a mode parameter. Use one `manage_ticket` tool with an action parameter. Retire a tool once a broader one covers its job.
- Prove it
- Track tool-choice accuracy on a scripted scenario set as the catalog grows (the trajectory layer on /evals/). Treat a drop as a signal to consolidate.
- Why
- A model cannot tell apart two tools named `search` from different systems in a flat list. It cannot know which `search` is Jira's and which is the internal wiki's.
- Build it
- Prefix every tool name with its system, such as `jira_search` and `wiki_search`. The name then shows ownership before the model reads the description.
- Prove it
- Grep the live tool list for name collisions or missing prefixes before each new integration ships. Add a lint check once the catalog has more than one source.
- Why
- A tool that reads on most calls but writes under one parameter combination gets called with read-level confidence and write-level consequences.
- Build it
- Split any tool whose action depends on a parameter value into two tools, one read and one write. Give each its own name and schema, and its own permission grant.
- Prove it
- Audit the catalog for any tool name that backs more than one permission scope. Treat each match as a design defect and fix it.
- Why
- Every tool definition sits in the context window on every turn. A forty-tool catalog is a permanent tax, even when the task calls four tools.
- Build it
- Cap the number of tools loaded per session. Select the subset at session start with an intent router or a fixed profile per task type.
- Prove it
- Measure the token share that tool definitions occupy at session start. Compare it with the task's own tokens, and alarm when it crosses a stated budget. See budget-the-window below.
02 Tool contracts (the schema)
The schema is the only briefing the model gets before it calls a tool. It has no comments and no onboarding doc. It has field names and types, plus the constraints the schema author chose to encode. If the schema is loose, the model fills the gap with a plausible guess, whether or not a real value exists. Tool callingTool calling lets a model request a structured action instead of only generating text. is only as reliable as this contract. You fix a hallucinated argument here, with an enum or a validated pattern. A better prompt does not fix it.
- Why
- A parameter named `user` leaves open whether it wants an ID, an email, or a display name. The model fills that gap with the form it saw most in training.
- Build it
- Name each parameter for the specific thing it holds: `user_id` instead of `user`, `start_date_iso` instead of `date`. Put the unit or format in the name.
- Prove it
- Read every parameter name with no description attached. If the name needs its description to be clear, rename it.
- Why
- A free-text field for a bounded value set invites the model to generate a plausible value. That is the same completion behavior that produces a hallucinated ID.
- Build it
- Constrain every field with a small, fixed value set to an enum. Validate server-side that each incoming value belongs to the set. Do not trust the schema to enforce itself.
- Prove it
- Compare the invalid-argument rate on enum fields with free-text fields (the tool-call-argument-check on /evals/). The gap should be large. If it is small, the enum constrains nothing.
- Why
- An optional field that the tool needs moves the failure downstream. The call passes the schema and fails at execution, with a worse error and less context.
- Build it
- Set `required` based on what the tool needs to act correctly. If a missing field always gives a degraded or wrong result, mark it required.
- Prove it
- Sample recent execution failures. Count how many trace back to a field that was optional in the schema but mandatory in practice. Fix each one in the schema.
- Why
- A description written for someone who knows the API assumes context the model lacks. State the units and formats, and how resources relate.
- Build it
- Write the description as you would brief a new hire. State what the tool does and what a valid input looks like. Name the units and formats it expects. Say how it relates to nearby tools.
- Prove it
- Give the schema and description alone to someone unfamiliar with the system. Ask them to predict a valid call. If they must guess, rewrite the description.
03 Tool responses (what comes back)
A tool response becomes the model's whole picture of what just happened. The model has no side channel to learn that a call failed. The response format must carry that signal. A raw database dump or a wall of UUIDs suits a human at a debugger. So does an HTTP 200 that wraps an error string. Each suits a model badly, because it reasons about the result on the next turn within a fixed token budget. A high-signal response lets the model notice a failure. A poor one lets the model report missing data as a fact.
- Why
- A response full of UUIDs gives the model a value to pass to the next call. It gives the model nothing to reason about. The model cannot tell two records apart by name.
- Build it
- Include a human-readable name or label beside every identifier in a response. The next reasoning step can then check its work against something legible.
- Prove it
- Read a sample of raw tool responses with no lookup table. Check that the meaningful content is legible. If every row is an ID, add a name field.
- Why
- A full record dump makes the model find the one relevant field on every call, and it spends tokens on all the other fields. The noise hides the signal.
- Build it
- Return only the fields the next decision needs by default. Add an explicit parameter to request more detail when a task calls for it.
- Prove it
- Measure average response size in tokens against the share of fields that a later step references (cost-per-completed-task on /evals/). Trim what nothing reads.
- Why
- An unbounded result set either blows the context budget on one call or gets truncated with no signal. In both cases the model does not know it holds a partial answer.
- Build it
- Set a sane default page size on every list-returning tool. When a response is truncated, say so. State how to narrow the next call, such as a filter to add or a cursor to pass.
- Prove it
- Trigger a truncated response on purpose. Confirm that the model, reading only the response, builds a correctly narrowed follow-up call.
- Why
- A stack trace or a generic 500 tells the model that something failed. It says nothing about the next step. The model repeats the identical call or gives up.
- Build it
- Return a specific error, such as 'no matching record for order 4471, check that the order exists'. Do not return an implementation-level exception. Give write tools an error path distinct from read tools.
- Prove it
- Fail a call on purpose. Confirm that the returned error alone lets the model correct its next attempt. If the model repeats the same call, the error taught nothing.
04 Context assembly & budget
Tool definitions and retrieved documents compete with prior turns for one fixed context window. Nothing makes that competition favor the task. Without management, the stable material (system instructions and tool schemas) drifts to the front. There, one reordered field breaks prompt caching. The volatile material, such as this turn's question, gets buried or evicted first. A team should set the context budget once, on purpose. A truncation strategy improvised on the day the window fills is a poor substitute.
Put static instructions and tool definitions first, and anything that changes per turn last.
Read-only- Why
- One volatile token near the front of a prompt breaks prompt caching for the whole prefix behind it. A timestamp, a reordered tool list, or a per-request ID can do this. A human reader sees an unchanged request.
- Build it
- Order every context assembly the same way. Put static system instructions and tool definitions first. Put retrieved content next. Put per-turn content last. Never reorder tools within a session.
- Prove it
- Check your cache-read share against provider usage data. Do not judge by how stable the prompt looks. Treat a low hit rate as an ordering bug.
- Why
- Without a stated allocation, every slice expands to fill the room. The fastest-growing slice starves the others. The task usually loses, because it is assembled last.
- Build it
- Write down a token allocation per slice before the system ships. The slices are instructions, tool definitions, retrieved content, history, and task. Alert when any slice crosses its share.
- Prove it
- Instrument token spend per slice on a sample of real sessions. Compare it with the written budget. Fix a slice at 3x its allocation before it causes an overflow.
- Why
- If you compact only when the window is full, eviction happens under pressure. The oldest or least structured content goes first. A long-running agent often needs exactly that state.
- Build it
- Define the compaction rule in advance. Summarize turns older than N steps, or compact when a slice crosses a stated share of budget. Run it on that schedule.
- Prove it
- Force a long run past the compaction trigger. Confirm that the agent's task state and constraints survive. If it forgets its goal, the rule cut the wrong content.
Mark retrieved content and tool output as data, and keep it structurally apart from instructions.
Writes- Why
- A model reads one undifferentiated stream of tokens. Nothing separates an instruction from retrieved text, so a document that reads like a command works like one.
- Build it
- Mark retrieved content and tool output as data wherever the format allows. Use a distinct field, or a wrapping delimiter that the system prompt names as untrusted. Never concatenate it into the instruction stream.
- Prove it
- Plant a labelled set of injection attempts in retrieved content and tool responses. Run it through the full tool-enabled path (the injection-red-team-set check on /evals/). Track resistance as its own number.
05 Permissions & blast radius
A tool call that can write or delete carries a different risk from one that only reads. If you treat them the same, you get one flat credential and one shared identity. The catalog then makes no distinction between them. Every prompt injectionPrompt injection is untrusted input crafted to override a model's system prompt or task. and hallucinated argument inherits the blast radius of the most dangerous tool in the session. This layer cannot stop a bad call from being attempted. It decides how far a bad call can reach before a human or a rollback steps in.
- Why
- One set of credentials for read and write means every read call carries the blast radius of a write it will never make. A hallucinated argument or a successful injection inherits permissions it did not need.
- Build it
- Grant read access by default. Make each write a separate tool with its own, narrower credential. Never use one shared scope that allows both.
- Prove it
- Audit the credential behind every tool. Confirm that a read-only tool's credential cannot perform a write at the provider level.
- Why
- The harness, a bounded-loop budget, or the model itself can retry a write call that timed out. Either the write happens twice, or the retry becomes the source of runaway spend.
- Build it
- Generate a unique idempotency key for each intended write. Pass the same key on every retry of that logical call. The receiving system then recognizes the repeat and does nothing.
- Prove it
- Retry a write call with the same key on purpose. Confirm that the second attempt does nothing. Then confirm that a new call with a new key succeeds.
- Why
- An autonomous agent has no backstop once a bad decision reaches an irreversible tool, such as a refund or an external message. The call executes as fast as a good one.
- Build it
- Require explicit human confirmation before any call that cannot be cleanly undone. Keep it until the surrounding controls have a track record. Enforce it in the harness, not in the prompt.
- Prove it
- Script an adversarial scenario that tries to make the agent issue a refund or send a message it should not. Confirm that the checkpoint fires. This is the tool-abuse-scenarios check on /evals/.
- Why
- In a flat tool list, a tool that deletes or sends data looks the same as a safe read. So does a tool that reaches an open-world resource. Nothing stops the model from treating it as casually as a lookup.
- Build it
- Add an explicit annotation to every destructive or irreversible tool definition. Annotate each tool that reaches outside a closed set of resources too. The harness and reviewers can then filter on it mechanically.
- Prove it
- Grep the live catalog for every write-capable or open-world tool. Confirm that each one carries the annotation. Fix any unmarked destructive tool before the next session loads it.
06 Transport & operations
A tool call that crosses a process boundary stops being a function call. The boundary can be a team's API or a vendor's MCP server. The call becomes a contract with an operations story. That story says who may call it and what happens when its shape changes. It also says how anyone reconstructs the events afterward. MCP standardizes the wire format for the crossing. It does not standardize the governance around it. Most failures at this layer come from an org that shipped the protocol and skipped the operations.
- Why
- A server, a transport layer, and an auth story add operational surface. If one process calls the tool, that surface guards a boundary that does not exist yet.
- Build it
- Keep the tool in-process as a plain function call in your own codebase. Move to MCP when it must cross a team or product boundary.
- Prove it
- Before you stand up an MCP server for a new tool, name the team or product that will call it. If the honest answer is 'nobody yet', the boundary has not arrived.
Version every tool contract, and give agents a real deprecation window before a breaking change.
Read-only- Why
- A schema that changes shape with no version marker breaks every agent pinned to the old shape. The break happens silently when the new version deploys. No compiler catches it.
- Build it
- Version every tool contract explicitly. Keep the old version live through a stated deprecation window. Require agents to pin a version instead of resolving to the newest.
- Prove it
- Ship a breaking change behind a new version. Confirm that an agent pinned to the old version keeps working through the whole deprecation window.
- Why
- A shared service account behind every agent in a fleet makes it impossible to answer 'which agent did this'. It also makes one agent's scope the scope of every agent.
- Build it
- Give every agent its own credential. Scope it to what that agent's tools need. Keep it in a vault, not in a prompt or a config file in source control.
- Prove it
- Pick any tool call in a trace. Confirm that you can name the agent identity that made it. The credential should show it, not the shared account or the human behind it.
Log every tool call as structured data: tool, arguments, outcome, latency, tokens, and cost.
Autonomous- Why
- Without a call-level log, a regression in cost, latency, or reliability is a vague complaint that the agent got slower. No call or version is available to trace it back to.
- Build it
- Log every field named in the rule on every call. Keep the log structured and queryable. This is the trace-first discipline that the handbook's operate stage applies to the rest of the system.
- Prove it
- Pick a random recent session. Reconstruct its full tool-call trace, cost included, from the logs alone. A missing call or an unattributed cost shows a gap in the logging.
No rules match that combination. Clear a filter to see more.
A rule in code
One rule bank entry, in a real schema.#
One tool definition, first wrong and then right, in the shape a model reads.
The same search tool, two ways.
A vague catalog entry and a narrow one, side by side. Only the right pane survives a session with forty other tools in it.
What breaks search-tool.json
{
"name": "search",
"description": "Searches for stuff.",
"parameters": {
"type": "object",
"properties": {
"options": { "type": "object" }
}
}
} A free-form name, a free-form options bag, no enum, and no cap on how many results come back.
What works search-tool.json
{
"name": "search_kb_by_tag",
"description": "Finds KB articles by tag. Not for fuzzy search.",
"parameters": {
"type": "object",
"properties": {
"tag": {
"type": "string",
"enum": ["billing", "auth", "deploy"]
},
"max_results": {
"type": "integer",
"default": 10,
"maximum": 50
}
},
"required": ["tag"]
},
"response_shape": {
"results": [
{ "id": "string", "title": "string" }
],
"truncated": "boolean"
}
} A narrow name, an enum for the one filter that matters, a bounded max_results, and a declared response shape.
- One job, one name a model can tell apart from its neighbors. One job per tool
- A closed set of tags beats a free-text field the model has to guess the values for. Enums over freetext
- A page size with a default and a ceiling, not an unbounded result set. Paginate with a hint
- The description states the case this tool is wrong for, not only the case it is right for. Describe it like a new hire
Five ways a tool interface fails
Five interface problems.#
Each failure has a cause on one of the surfaces above. Each has a named symptom that this site documents elsewhere.
Catalog creep
Looks like: Two or three tools in one session do nearly the same thing under slightly different names. The model picks the one it saw most often in training. The task may have needed another.
Guardrail: Audit the catalog for near-duplicates before you add a tool. Merge or retire the older one. Do not ship a fourth way to make the same call.
Failure mode: Tool argument hallucinationContext crowd-out
Looks like: Tool definitions and retrieved results together fill most of the window before the model reaches the question. When something must give, the task is the first thing truncated.
Guardrail: Budget context per slice: instructions, tool definitions, retrieved content, and task. Alarm when a slice crosses its share. Do not discover the dropped task after the fact.
Failure mode: Context overflow drops the taskCheerful failure
Looks like: A tool call returns HTTP 200 with an empty or malformed body. The model reports the missing data as a fact. It does not flag a failure.
Guardrail: Give every response a typed status field that the model can branch on: ok, empty, or error. A failure must then appear as a failure. The model does not infer it from a blank body.
Failure mode: Silent tool failureRetry storm
Looks like: A write tool call times out and retries automatically. The mutation then happens twice, or the retry itself burns the budget.
Guardrail: Put an idempotencyAn idempotent operation produces the same result no matter how many times it runs. key on every write call, so a retried request is provably a no-op. Cap retries explicitly. Never leave the ceiling unbounded.
Failure mode: Retry-amplified spendInstructions in the payload
Looks like: A tool response includes retrieved text that reads like an instruction. The model has no structural way to tell data from command, so it follows the text.
Guardrail: Treat everything a tool returns as data. Scope every tool to least privilege, so a followed instruction has nowhere dangerous to go.
Failure mode: Injection via retrieved content What this is built from
Every source is on this site.#
This page cites no benchmark numbers and no client names. It cites only what this site documents.
- This site documents the typed tool contractA tool contract is the schema a tool exposes to a model. and the bounded agent loop in full. This page joins their design rules into one interface. Pattern: Typed tool contract
- This site names five failure modes, and the cards above link to them. Each one is the reason a rule above exists. Failure mode: Silent tool failure
- CodeHerder runs its own tool catalog and context budget across a fleet of coding agents. That applies the catalog, context, and permissions surfaces above at fleet scale. CodeHerder
- We are building CostMon to answer the per-call cost line of the transport layer (log every call, above). It will give teams a self-serve number in place of a monthly reconciliation project. CostMon (coming soon)
- The Model Context Protocol glossary entry covers the standard behind this page's transport surface. Adopt it deliberately. Glossary: Model Context Protocol (MCP)
Sources
- Model Context Protocol - MCP Specification: Tools (2026-07-28)
The specification text our tool-design and catalogue rules are checked against. Retrieved 2026-08-24.
- JSON Schema - JSON Schema, Draft 2020-12
The schema specification a tool description must satisfy under our rule bank. Retrieved 2026-08-24.
Which surface causes it?
A Ship Audit checks the tool interface your agent runs against: catalog, contract, permissions, and the rest. It tells you which surface is the live risk.
Questions
Before you write the schema.#
What teams ask us before they scope a tool catalog.
01 How many tools is too many?
No fixed number applies. The failure mode is near-identical tools crowding the model's choice. Audit for overlap before you add a tool. Cap what loads per session to what the task needs.
Link to this answer: How many tools is too many?02 Do we need MCP, or is in-process enough?
Stay in-process until a tool crosses a team or product boundary. MCP costs a server, a transport layer, and an auth story. That cost pays off once another team or product needs to call the same tool. Earlier, it adds operational surface with no boundary to justify it.
Link to this answer: Do we need MCP, or is in-process enough?03 Which rule should we start with?
Start with read-only by default, and add a typed status field to every response. Together they close the two failure modes with the most early damage. One is an over-scoped credential. The other is a silent failure that the model reports as fact.
Link to this answer: Which rule should we start with?04 How does this relate to the Eval Stack?
The Eval Stack tells you whether the system works. This page covers the interface the agent works through. A contract check on /evals/ scores a tool contract built to the rules above. The two complement each other.
Link to this answer: How does this relate to the Eval Stack?05 Is this just prompt engineering?
No. Prompt engineering shapes what you ask the model. This page covers the surface the model acts through once it decides to act. A well-designed tool catalog survives a mediocre prompt. A great prompt on a badly designed tool interface still fails at the tool call.
Link to this answer: Is this just prompt engineering?06 What is context engineering, and how does it differ from prompt engineering?
Context engineering decides what occupies the token window at every step, and in what order. That includes tool definitions, retrieved content, history, and the task itself. Prompt engineering is one input to it. See the context surface above.
Link to this answer: What is context engineering, and how does it differ from prompt engineering?