How do you let an agent act without letting it do something you can't undo?
Every tool the agent can call is scoped to the narrowest permission the task genuinely needs, and any action that can't be cleanly undone — a refund, a delete, an external message — requires an explicit human confirmation before it executes, not just a plausible-looking model decision. The two controls compound: least privilege bounds the blast radius of anything that slips through, and the checkpoint stops the worst actions from executing at all.
Also known as approval gate, human-in-the-loop confirmation, least-privilege tool scope
What problem does this solve?
A model reads one undifferentiated stream of tokens with no privileged channel marking some text as trusted instruction and other text as data. Retrieved content — a document, a ticket, a webpage — enters that same stream, so text crafted, or accidentally shaped, to read like a command has no structural reason to be treated differently from an actual instruction. An agent with broad tool access and no gate in front of irreversible actions has no backstop when that happens.
Every additional tool call an agent can make widens the set of places an injected or simply wrong instruction can reach something real — a refund tool, an email sender, a delete action. A successful injection or a bad model decision early in a session also persists in context for every later step, so the risk does not reset each turn; it accumulates.
How does it work?
Every tool is scoped with an explicit allowlist of what it is permitted to do, sized to exactly what the current task needs, not the union of everything the agent might ever plausibly need, which is the default that broad tool access tends toward. A narrower grant means a bad decision, whatever caused it, has a small blast radius by construction, not merely by policy the model is trusted not to misuse.
Separately, any action that cannot be cleanly reversed — a refund, a permanent delete, a message sent externally, anything where "undo" means a manual cleanup — requires a human to confirm before it executes, regardless of how confident the agent's reasoning looked. This is not a review of the agent's general competence; it is a gate specifically on the subset of actions where a mistake is expensive to unwind.
Model output that feeds a later tool call or another model is treated as untrusted input at that boundary, validated the way an anonymous user's request would be, because a successful injection functionally makes it exactly that. Over time, as a specific action type accumulates a track record of correct, low-risk decisions, the checkpoint for that specific action, not the whole system, can be relaxed deliberately, not by default.
What are the moving parts?
- Scope every tool to the narrowest permission needed
Grant exactly what the current task requires, not the broadest set it might ever plausibly need.
- Classify every tool call by reversibility at design time
Decide up front which actions are reversible and which require confirmation.
- Gate irreversible actions on human confirmation
Require explicit approval before execution, regardless of how confident the agent looked.
- Treat cross-boundary model output as untrusted
Validate output feeding another tool or model the way an anonymous request would be validated.
- Log every confirmation request and its outcome
Track approved, rejected, and timed-out requests as structured data.
- Relax a specific action's gate only deliberately
Loosen a checkpoint after a measured track record for that action type, never by default.
When does it fit, and when doesn't it?
Use it when
- The agent has any reachable tool that can mutate state in a way that's costly or impossible to reverse (refunds, deletes, external sends)
- The agent's inputs include retrieved or third-party content (documents, tickets, scraped pages) alongside direct user instructions
- You're early in production rollout and don't yet have a track record to justify full autonomy on a given action
- A wrong action's cost — financial, reputational, compliance — is materially higher than the cost of a brief human delay
Don't use it when
- Every tool the agent can reach is read-only or trivially reversible — there is no irreversible action to gate
- The approval queue has no real owner or SLA, so 'requires confirmation' becomes 'sits unactioned forever' — fix the queue before adding the gate, or it just becomes a silent drop
- The task is fully offline or batch with no plausible attacker-controlled or untrusted input reaching the agent at all, and the action, while technically irreversible, is genuinely low-stakes (e.g. deleting a scratch file)
What does it actually cost?
A successful injection or a wrong model decision has a small, bounded blast radius by construction
Least-privilege scoping means more distinct tool definitions to build and maintain instead of one broad, flexible tool
Irreversible actions can't execute on a plausible-looking but wrong decision alone
Adds latency and a real operational queue — someone has to actually staff and answer the confirmation requests
Treating cross-boundary model output as untrusted catches injected instructions before they reach a second tool
Extra validation logic at every hop where one model's output feeds another call, which is genuine engineering surface
A deliberate, evidence-based relaxation process lets autonomy grow safely over time
Requires actually tracking outcomes per action type to earn that relaxation — skipping the tracking and relaxing anyway defeats the entire pattern
How do you know it's working?
- Confirmation request volume and resolution time (approved/rejected/timed out), per action type
- Rate of confirmation requests reaching a real human vs. timing out unactioned
- Reachable-tool audit: which tools each agent instance can call, and whether any exceed the current task's actual need
- Injection eval pass rate — planted red-team documents that shouldn't trigger an unconfirmed action
What failure modes does this prevent?
A Ship Audit checks which of these patterns your system actually needs, prioritized against what's most likely to break first.