A coding-agent delivery fleet
A control plane for running many coding agents at once without them colliding on the same files or losing track of what any of it costs.
“We're running several coding agents already, and it's turning into a wall of terminal tabs with no idea what any of it actually costs.”
What does the system look like?#
How does a request move through it?#
- A task enters the registry
Work is defined once, in a shared registry, rather than assigned informally across a set of terminal sessions nobody else can see into.
- An agent claims it without colliding
Claiming a task checks what else is already in flight against that repo and file set, so two agents on different machines can never be handed the same file at the same time.
- The agent works inside its own bounded loop
Each agent runs under the same step, token, and time budget discipline as any other bounded agent loop, in its own branch or worktree.
- Cost rolls up per task, not per machine
Every run reports its own spend back to the task it was assigned, so cost is a query by task, repo, or agent, not a reconstruction from a provider invoice.
- CI gates before a human ever looks
Lint, tests, and eval checks run before the diff reaches a reviewer; a failure routes back to the agent, not to a person's queue.
- A human reviews, then merge triggers verify
A person still reviews the diff; merging kicks off a verify stage that checks the change actually shipped what it claimed, closing the loop back into the registry and the delivery record.
What are the pieces, and what breaks without each one?#
Task registry
Hold the single, shared definition of what work exists and its current state, independent of which machine an agent runs on.
Breaks without it: Visibility into the fleet degrades into a wall of terminal tabs with no shared view of what any agent is actually doing.
Collision-free claiming
Check what's already in flight against a repo and file set before handing a task to an agent.
Breaks without it: Two agents on different machines can be handed the same file at the same time, and the first sign of it is a clobbered edit or a merge conflict.
Bounded coding agent
Work one task inside its own branch or worktree, under an explicit step and time budget.
Breaks without it: A single stuck agent can burn far more compute and time than the task ever warranted, with nothing forcing it to stop.
Per-task cost ledger
Attribute every run's spend back to the task it was assigned, rolled up by task, repo, or agent.
Breaks without it: Spend is scattered exactly like visibility is: no single place to see what a task actually cost until the invoice lands.
CI gate
Run lint, tests, and eval checks on every agent-produced diff before a human ever sees it.
Breaks without it: A reviewer's time goes to catching mechanical problems a gate should have caught automatically, instead of judging the actual design decision.
Human review & verify
Review the diff, then confirm after merge that the change actually shipped what it claimed.
Breaks without it: A pile of unreviewed agent-produced PRs is not delivery; it is a bigger, less-trusted backlog than the one this replaces.
Where does this need a decision, not a default?#
| Decision | Default choice | Why |
|---|---|---|
| Fleet control plane | Adopt an existing task registry and coordination layer rather than build your own from scratch | Collision-free tasking across machines with no shared filesystem is genuinely hard to get right; CodeHerder is our own answer to it, one option among several, not the only shape that works. |
| CI and test infrastructure | Extend the existing pipeline with agent-specific gates rather than stand up a parallel one | A second, separate pipeline for agent-produced code just means two things to keep in sync instead of one gate everyone trusts. |
| Review process | Human review on every PR before merge, not a sampled review, until the fleet has a real track record | Sampling review before you have evidence of what a given agent gets wrong and how often is a way to discover the gap from a production incident instead of a code review comment. |
| Cost accounting | A per-task ledger rather than per-machine or per-seat billing rolled up after the fact | Per-machine billing tells you what the fleet cost in total; it does not tell you which task, or which agent, is the one actually driving the number up. |
What actually drives the bill?#
- Agent compute and token spend per task
- CI runs per task, including retries on a flaky or failing gate
- Human review time per PR, which does not shrink just because an agent wrote the diff
Dominant cost Agent compute and token spend once a fleet runs many tasks concurrently, easily lost inside one undifferentiated provider invoice without per-task attribution.
The lever A per-task cost ledger to see where spend actually concentrates, paired with the same bounded-loop discipline that caps any single agent's worst case.
Model your own numbers with the AI Cost Calculator →How do you know it actually works?#
Golden-set gate in CI on every agent-produced change
The same regression gate a human-written change would get, applied without exception to agent output.
Attributes spend to the task that drove it, not to an average across the whole fleet.
Terminal-state rate across the fleet
Tracks what share of agent runs actually finish in a defined state versus stalling or looping.
What will your reviewers ask about this?#
A step, token, and wall-clock ceiling per task, the same discipline any bounded agent loop needs, applied per agent in the fleet.
Each agent holds only the repository and tool access its current task needs, not a standing credential for the whole fleet.
Named accountable owner per task
Every task has a specific person accountable for what merges, not "the fleet" as an undifferentiated whole.
Which patterns and failure modes tie in?#
How long does a first version actually take?#
3-6 weeks to run one small fleet on one repository end to end, from task claim to merged PR; longer once collision-free claiming has to hold across multiple repos and machines.
When is this the wrong shape?#
- One or two engineers already run one agent each with no collision problems between them. A fleet control plane solves a coordination problem you do not have yet.
- Nobody is willing to review the diffs a fleet produces. A pile of unreviewed agent-produced PRs is a bigger backlog, not delivery.
- Your CI is not fast or reliable enough to gate a task before a human sees it. Fix that first, or every agent-produced PR just adds noise to a gate that already leaks.
Sources
- Google DORA - DORA Research
The research programme behind the delivery metrics our release and delivery pages measure. Retrieved 2026-08-24.
- Google DORA - DORA Capabilities Catalog
The capability list this research associates with high-performing delivery teams. Retrieved 2026-08-24.
A Ship Audit checks which of these an existing or planned system actually needs, against what is most likely to break first.