A coding-agent delivery fleet
A control plane for running many coding agents at once. The agents do not collide on the same files. You can see what each one costs.
“We already run several coding agents. It has turned into a wall of terminal tabs. We have no idea what it costs.”
What does the system look like?#
How does a request move through it?#
- A task enters the registry
Work is defined once, in a shared registry. It is not assigned informally across terminal sessions that nobody else can see.
- An agent claims it without colliding
Claiming a task checks what is already in flight against that repo and file set. Two agents on different machines never get the same file at the same time.
- The agent works inside its own bounded loop
Each agent runs in its own branch or worktree. It follows the same step, token, and time budget discipline as any other bounded agent loop.
- Cost rolls up per task
Every run reports its spend back to the task it served. Cost becomes a query by task, repo, or agent. Nobody rebuilds it from a provider invoice.
- CI gates before a human ever looks
Lint, tests, and eval checks run before the diff reaches a reviewer. A failure routes back to the agent. It does not enter a person's queue.
- A human reviews, then merge triggers verify
A person still reviews the diff. Merging starts a verify stage. That stage checks that the change shipped what it claimed. The result closes the loop in the registry and the delivery record.
What are the pieces, and what breaks without each one?#
Task registry
Hold the single, shared definition of what work exists and its current state. It does not depend on the machine an agent runs on.
Breaks without it: Visibility into the fleet degrades into a wall of terminal tabs. Nobody has a shared view of what each agent does.
Collision-free claiming
Check what is already in flight against a repo and file set. Do this before you hand a task to an agent.
Breaks without it: Two agents on different machines can get the same file at the same time. The first sign is a clobbered edit or a merge conflict.
Bounded coding agent
Work one task inside its own branch or worktree, under an explicit step and time budget.
Breaks without it: One stuck agent can burn far more compute and time than the task warrants. Nothing forces it to stop.
Per-task cost ledger
Attribute every run's spend to its task. Roll the spend up by task, repo, or agent.
Breaks without it: Spend is as scattered as visibility. No single place shows what a task cost until the invoice lands.
CI gate
Run lint, tests, and eval checks on every agent-produced diff. A human sees the diff only after the gate passes.
Breaks without it: Reviewer time goes to mechanical problems that a gate should catch. The reviewer has less time to judge the design decision.
Human review & verify
Review the diff. After merge, confirm that the change shipped what it claimed.
Breaks without it: A pile of unreviewed agent-produced PRs is not delivery. It is a bigger, less-trusted backlog than the one it replaces.
Which choices need a decision?#
| Decision | Default choice | Why |
|---|---|---|
| Fleet control plane | Adopt an existing task registry and coordination layer. Do not build your own from scratch | Collision-free tasking across machines with no shared filesystem is hard to get right. CodeHerder is our own answer to it. It is one option among several. |
| CI and test infrastructure | Extend the existing pipeline with agent-specific gates. Do not stand up a parallel one | A second, separate pipeline for agent-produced code means two things to keep in sync. One gate that everyone trusts is better. |
| Review process | Human review on every PR before merge. Use a sampled review only after the fleet has a real track record | Sampled review needs evidence of what an agent gets wrong and how often. Without that evidence, you find the gap in a production incident. A code review comment would have found it sooner. |
| Cost accounting | A per-task ledger. Avoid per-machine or per-seat billing rolled up after the fact | Per-machine billing tells you what the fleet cost in total. It does not tell you which task or agent drives the number up. |
What drives the bill?#
- Agent compute and token spend per task
- CI runs per task, including retries on a flaky or failing gate
- Human review time per PR. It does not shrink when an agent writes the diff
Dominant cost Agent compute and token spend, once a fleet runs many tasks at the same time. Without per-task attribution, one undifferentiated provider invoice hides it.
The lever A per-task cost ledger shows where spend concentrates. The same bounded-loop discipline caps any single agent's worst case.
Model your own numbers with the AI Cost Calculator →How do you know it works?#
Golden-set gate in CI on every agent-produced change
The same regression gate that a human-written change gets. Apply it to all agent output.
Attributes spend to the task that drove it. An average across the fleet hides that.
Terminal-state rate across the fleet
Tracks what share of agent runs finish in a defined state. The rest stall or loop.
What will your reviewers ask about this?#
A step, token, and wall-clock ceiling per task. Any bounded agent loop needs this discipline. Apply it to each agent in the fleet.
Each agent holds only the repository and tool access that its current task needs. No standing credential covers the whole fleet.
Named accountable owner per task
Every task has a specific person accountable for what merges. "The fleet" is not a person.
Which patterns and failure modes tie in?#
How long does a first version take?#
3-6 weeks to run one small fleet on one repository end to end, from task claim to merged PR. Plan for longer once collision-free claiming must hold across multiple repos and machines.
When is this the wrong shape?#
- One or two engineers already run one agent each. They have no collision problems. A fleet control plane solves a coordination problem you do not have yet.
- Nobody is willing to review the diffs a fleet produces. A pile of unreviewed agent-produced PRs is a bigger backlog. It is not delivery.
- Your CI is not fast or reliable enough to gate a task before a human sees it. Fix that first. Otherwise every agent-produced PR adds noise to a gate that already leaks.
Sources
- Google DORA - DORA Research
The research programme behind the delivery metrics our release and delivery pages measure. Retrieved 2026-08-24.
- Google DORA - DORA Capabilities Catalog
The capability list this research associates with high-performing delivery teams. Retrieved 2026-08-24.
A Ship Audit checks which of these an existing or planned system needs. It ranks them by what is most likely to break first.