Prompt Injection Is a Data Problem: A Threat Model You Can Ship Against
Prompt injection can't be filtered away: the model can't reliably tell instructions from data. Here's the actual threat model and the controls that hold up.
Read → Topics
Shipping to Production
The gap between a working notebook and a system that runs reliably, observably, and cheaply at scale is usually a bigger lift than the original prototype. This is our work on closing that gap.
Insights
Why Your AI Agent Works in the Demo and Stalls in Production
An agent that nails the demo stalls in production because reliability compounds across steps. Here's the math, the real failure modes, and how to ship one anyway.
Read →What AI Actually Costs in Production, by Workload
A modeled, reproducible benchmark of cost-per-successful-outcome across four common AI workloads, with every token assumption, price, and overhead multiplier shown.
Read →Your RAG Demo Lied to You
Retrieval that looks flawless on ten clean PDFs falls apart on a real corpus. Here's why, and what evaluating retrieval quality actually requires.
Read →What an AI Feature Actually Costs in Production
Token costs that look trivial in a demo compound fast at scale. Here's how to make cost a first-class metric instead of a surprise on the invoice.
Read →From Notebook to Production: What Actually Breaks
The gap between a working AI demo and a production feature is auth, latency, cost and fallbacks. Here's how we close it without a rewrite.
Read →Case studies
From Notebook to Production: An Extraction Model You Could Trust
A mid-market healthtech had a document-extraction model that dazzled in a notebook and hallucinated on real intake traffic. An eval harness and confidence gating took hallucinations from 12% to 3% and shipped it in 8 weeks.
Read →Failure modes
Stale index serves deleted content
Your retrieval index was built once at ingest and never told the source changed. When a document is edited or deleted, nothing re-embeds the new version or tombstones the old chunk, so the stale vector keeps scoring well and keeps getting served — confidently, and with no signal to the reader that it is out of date.
Read →Unbounded agent loop
The agent has no step budget, no token budget, and no way to recognize it is stuck — a failing tool call stays in its context and keeps looking like a reasonable next thing to try, so it keeps trying variations of the same failed approach until something external (a timeout, a bill, a human) stops it.
Read →Tool argument hallucination
Loose tool schemas — free-form string IDs, everything optional — give the model room to fill a gap with something plausible-looking instead of something real, and with no server-side validation catching the mismatch before execution, a confidently invented ID reaches a system that expects a real one.
Read →Silent tool failure
The tool returned HTTP 200 with an error message in the body, or an empty result set, and the agent read the absence of data as evidence rather than as a failure — because nothing in the response forced a distinction between "nothing matched" and "something broke."
Read →Context overflow drops the task
As the conversation grows, a naive truncation strategy drops the oldest messages to stay under the context window — and the oldest messages are exactly where the system prompt and the original task state usually live, so the agent keeps running with no memory of what it was actually supposed to do.
Read →Injection via retrieved content
Yes — retrieved content arrives on the same channel as instructions, so a document, ticket, or webpage crafted (or compromised) to contain commands can have the model execute them with its real tool permissions, and the system has no built-in way to tell 'instruction from us' apart from 'text we retrieved.'
Read →Patterns
Intent router to specialists
A cheap, fast classifier reads the incoming request first and routes it to one of several narrow, single-purpose agents — each holding only the tools, context, and instructions its job needs — instead of a single god-agent carrying every tool definition and every rule for every possible request. The router's only job is picking the right specialist; the specialist's only job is the task it was built for.
Read →Bounded agent loop
An agent loop runs under an explicit budget — a maximum step count, a token ceiling, and a wall-clock limit — plus a termination contract that forces every run to end in one of a small number of named states: success, failure, or escalation. When the budget is exhausted before the model reaches one of those states on its own, the harness ends the run itself rather than letting it continue indefinitely.
Read →Typed tool contract
Every tool argument is defined by a strict JSON schema — enums for known value sets, validated patterns for IDs, required fields wherever the tool genuinely needs them — with no free-text catch-all surface. A call that fails validation is rejected with a structured, actionable error the model can act on, never silently coerced or passed through to execution.
Read →Human checkpoint before irreversible actions
Every tool the agent can call is scoped to the narrowest permission the task genuinely needs, and any action that can't be cleanly undone — a refund, a delete, an external message — requires an explicit human confirmation before it executes, not just a plausible-looking model decision. The two controls compound: least privilege bounds the blast radius of anything that slips through, and the checkpoint stops the worst actions from executing at all.
Read →