The Agentic Delivery Playbook
The operating model for delivery with a fleet of coding agents. It has six control surfaces and 22 practices, plus the metrics that catch review debt early.
26 min read Read → Topics
Shipping to Production
The gap between a working notebook and a system that runs reliably and cheaply at scale is usually a bigger lift than the original prototype. This is our work on closing that gap.
Start with The AI Release Path if you own shipping AI to production. It's for engineers who need a reliable way to ship a change and reverse it fast.
Guides#
The Agent Tool Interface
Agents fail at the tool interface more often than at the model. The design rules for building an interface that holds up in production.
25 min read Read →The AI Release Path
How to ship a change to an AI system, ramp it safely, and reverse it fast if it turns out to be wrong.
34 min read Read →The Agent Adoption Playbook
The pilot worked. Here is how the rest of your engineers start using it, and the numbers that separate real adoption from seat activation.
33 min read Read →MCP in Production
MCPMCP is an open standard for connecting LLM applications to tools and data sources. standardizes the wire format. Identity, the catalog and the trust boundary are still yours to build, plus a ten-question gate for a new server.
42 min read Read →Architectures#
Agentic workflow runner for a back-office queue
An agent that works a queue of cases end to end. It runs in a bounded loop. A human checkpoint covers any action it cannot cleanly undo.
7 min read Read →Document extraction with a human review loop
Turn inbound documents into structured records. A person reviews only the cases where the model is unsure.
6 min read Read →An assistant inside a product you already ship
An assistant embedded in a product you already ship. It is scoped to the signed-in user's own data and permissions.
6 min read Read →A coding-agent delivery fleet
A control plane for running many coding agents at once. The agents do not collide on the same files. You can see what each one costs.
6 min read Read →Comparisons#
Agents vs Pipelines: When Autonomy Is Worth the Reliability Cost
Agentic autonomy and deterministic pipelines aren't a spectrum of ambition. They trade flexibility against reliability, and your workflow decides which.
7 min read Read →Insights#
What Our Coding Agents Did to Get Past the Build Checks
A coding agent optimises for the check and skips the goal. Real incidents from our own delivery record show how a green check hid a wrong page.
9 min read Read →Prompt Injection Is a Data Problem: A Threat Model You Can Ship Against
Prompt injectionPrompt injection is untrusted input crafted to override a model's system prompt or task. can't be filtered away: the model can't reliably tell instructions from data. This is the threat model and the controls that hold up.
10 min read Read →Why Your AI Agent Works in the Demo and Stalls in Production
An agent that nails the demo stalls in production because reliability compounds across steps. We cover the math, the failure modes, and how to ship anyway.
8 min read Read →AI Cost per Outcome in Production, by Workload
A reproducible benchmark of cost per successful outcome across four common AI workloads, with every token assumption, price, and overhead multiplier shown.
9 min read Read →Your RAG Demo Lied to You
Retrieval that looks flawless on ten clean PDFs falls apart on a real corpus. We explain why, and what a retrieval eval requires.
7 min read Read →What an AI Feature Costs in Production
Token costs that look trivial in a demo compound fast at scale. Make cost a first-class metric instead of a surprise on the invoice.
7 min read Read →From Notebook to Production: What Breaks
The gap between a working AI demo and a production feature is auth, latency, cost and fallbacks. We close it without a rewrite.
7 min read Read →Case studies#
From Notebook to Production: An Extraction Model You Could Trust
A healthtech's extraction model dazzled in a notebook, then hallucinated on real traffic. An eval harness cut hallucinationsA hallucination is a confident, fluent output that is factually wrong or unsupported. from 12% to 3% in 8 weeks.
6 min read Read →Patterns#
Intent router to specialists
A fast, cheap classifier routes each request to one narrow specialist agent. Each specialist holds only the tools and instructions its job needs. The router picks the specialist. The specialist does the task.
Read →Bounded agent loop
A bounded agent loop runs under an explicit step budget and token ceiling. A wall-clock limit also applies. A termination contract forces every run to end in a small set of named states, such as success or escalation. When the budget runs out first, the harness ends the run itself.
Read →Typed tool contract
A strict JSON schema defines every tool argument. A call that fails validation gets a structured, actionable error. The schema uses enums for known value sets and validated patterns for IDs. It marks fields required where needed and offers no free-text catch-all. The system never coerces a failed call or passes it to execution.
Read →Human checkpoint before irreversible actions
Each tool gets the narrowest permission its task needs. A human confirms any action that cannot be undone before it runs. Examples are a refund or an external message. A plausible-looking model decision is never enough on its own. The two controls work together. Least privilege limits the blast radius of anything that slips through. The checkpoint stops the worst actions from running at all.
Read →Failure modes#
Stale index serves deleted content
Your retrieval index was built once at ingest and never told the source changed. When a document is edited or deleted, nothing re-embeds the new version or tombstones the old chunk. The stale vector keeps scoring well and keeps getting served with confidence, giving the reader no signal that it is out of date.
Read →Unbounded agent loop
The agent has no step budget, no token budget, and no way to recognize it is stuck. A failing tool call stays in its context and keeps looking like a reasonable next try. The agent keeps trying variations of the same failed approach. Something external, such as a timeout or a human, finally stops it.
Read →Tool argument hallucination
Loose tool schemas (free-form string IDs, everything optional) give the model room to fill a gap with something plausible-looking instead of something real. With no server-side validation catching the mismatch before execution, a confidently invented ID reaches a system that expects a real one.
Read →Silent tool failure
The tool returned HTTP 200 with an error message in the body, or an empty result set. The agent treated the absence of data as evidence and missed the failure. Nothing in the response forced a distinction between "nothing matched" and "something broke."
Read →Context overflow drops the task
As the conversation grows, a naive truncation strategy drops the oldest messages to stay under the context window. The oldest messages usually hold the system prompt and the original task state. The agent keeps running with no memory of what it was supposed to do.
Read →Injection via retrieved content
Yes. Retrieved content arrives on the same channel as instructions. A document, ticket, or webpage crafted or compromised to contain commands can make the model execute them with its real tool permissions. The system has no built-in way to tell 'instruction from us' apart from 'text we retrieved.'
Read →Tools#
AI Readiness Scorecard
A fast, honest read on whether your data, infra, and process are ready to ship AI.
Read →