What AI Actually Costs in Production, by Workload

Cost per successful outcome ranges from a fraction of a cent for a single RAG Q&A call on a small model to roughly sixteen cents for a multi-step agentic tool loop on a frontier model. Across four common workloads, the modeled gap between a clean-call sticker price and an actual successful outcome runs from about 1.2x to more than 3x once retries, growing context, and failed attempts are priced in. These are modeled numbers, not measured client telemetry.

Every price, token count, and multiplier below is stated so you can rederive each number yourself. This is the concrete counterpart to what AI actually costs in production: that piece explains why the gap exists; this one puts numbers on it.

How we modeled cost per successful outcome

The unit that matters is cost per successful outcome, not cost per call, because a call that fails still costs money and something else has to pay for it. The formula:

cost_per_successful_outcome = (input_tokens × input_price + output_tokens × output_price) × overhead_multiplier ÷ success_rate

input_tokens × input_price + output_tokens × output_price is the clean-call sticker price: what one attempt costs if it works the first time. The overhead_multiplier folds in retries, fallback double-billing, and the extra tokens a growing context re-sends on every turn. Dividing by success_rate is what turns a per-call price into a per-outcome price: if four out of five attempts succeed, the successful ones have to absorb the cost of the one that didn’t.

Prices (public list prices as of 2026-07-11; representative tiers, not a specific vendor SKU, since list prices move):

• A small/fast model tier ≈ $0.15 / $0.60 per 1M input/output tokens.
• A frontier model tier ≈ $3 / $15 per 1M input/output tokens.

Per-workload assumptions used to compute every cell below:

• RAG Q&A: one call, ~4,000 input tokens (retrieved chunks + query) + ~400 output tokens; overhead 1.1x; success rate 95%.
• Document extraction: one call, ~8,000 input tokens (one document) + ~800 output tokens (structured JSON); overhead 1.25x (reparse/repair passes); success rate 90%.
• Chat with history: one 10-turn session modeled as a single outcome, input growing linearly from ~500 to ~5,000 tokens per turn (27,500 tokens total) + 500 output tokens per turn (5,000 total); overhead 1.15x; success rate 92% for the whole session.
• Agentic tool loop: five chained model calls per outcome, ~2,000 input + ~300 output tokens each (10,000 in / 1,500 out total) as context grows across the chain; overhead 1.4x; success rate 85% per step, which compounds to roughly 44% for the whole five-call outcome (0.85⁵).

What each workload costs

Prices as of 2026-07-11; modeled from the assumptions above, not measured client telemetry.

A single RAG answer on a small model is genuinely cheap either way — a tenth of a cent whether you count the clean call or the successful outcome. Document extraction and chat sessions cost more mainly because they push more tokens, not because the multiplier is dramatic. The agentic tool loop is the outlier. Its modeled cost is more than three times its sticker price, because chaining five calls means chaining five chances to fail.

Why the modeled cost runs higher than the sticker price

The gap between a clean-call sticker price and a modeled cost per successful outcome comes from exactly two places, and they don’t contribute equally.

The overhead multiplier (1.1x to 1.4x here) covers retries on malformed or rate-limited calls, a fallback provider getting billed on top of the original attempt, and the extra tokens a growing context re-sends every turn. It’s a real cost, but on its own it’s a modest tax: 10% to 40% in this model.

The success-rate divisor is where multi-step workloads get expensive, because failure compounds. RAG Q&A and document extraction are single calls, so their stated success rate applies once. The agentic tool loop chains five calls, and if each step only succeeds 85% of the time independently, the chain as a whole succeeds roughly 0.85⁵ ≈ 44% of the time. More than half of all attempts fail somewhere along the way, and the outcomes that do succeed have to cover the cost of all of them. That’s why the agentic loop’s modeled cost runs about 3.2x its sticker price while RAG Q&A’s runs about 1.2x, even though its per-step overhead and success numbers don’t look dramatically worse in isolation. Cost adds with each chained call; failure multiplies.

This is the same mechanism the qualitative piece on AI cost describes: retries, fallbacks, and growing context billing twice, modeled here with actual numbers instead of as a general warning. The wider 5x–20x gaps teams report in the wild usually stack a second effect on top of this one: production prompts running far longer than whatever they benchmarked against the pricing page in the first place.

What this means for a feature you’re about to ship

Before you pick a model tier, model the workload shape, not just the per-token price. A frontier model looks like a 20x price jump over a small/fast one on the pricing page. But for a single RAG call, that’s the difference between $0.00097 and $0.0208 per outcome, both trivial at almost any volume. The decision that actually matters is how many chained calls an outcome needs and how reliable each one is, because that’s what turns a modest overhead multiplier into a 3x cost multiplier. A tool loop that’s cheap to call once but calls itself five times per outcome, at 85% reliability per step, is more expensive than its sticker price suggests by a wider margin than switching model tiers ever will be.

Run your own workload’s numbers through the same formula: plug in your token counts, tier, and expected success rate at the cost calculator before you commit to an architecture, not after the first month of invoices.

For the specific mechanisms behind a real bill running ahead of the modeled one, see the field guide entries on retry-amplified spend and prompt cache never hits.