# Everything we've published. Source: https://customlabs.io/library/ Updated: 2026-09-14 Library # Everything we've published. 142 entries across 11 sections: every insight, case study, comparison, pattern, failure mode, and reference page in one browsable index. Every entry below has a Markdown twin. Add `.md` to its URL to read the plain version. See [/llms.txt](https://customlabs.io/llms.txt) for the full rule. Start here Architectures Insights Comparisons Case studies Patterns Failure modes Tools & diagnostics Topics Glossary The studio Showing all 142 entries Clear filters ## 01 Start here The twelve flagship guides. Read one front to back before diving into the reference material below. ### The Applied AI Handbook A six-stage handbook for shipping applied AI: decide, design, build, evaluate, operate, cost, each linked to the reference material behind it. Sep 2026 ### 01 Decide Should we build this with AI at all: build, buy, or skip? Sep 2026 · 7 min read ### 02 Design What shape is the system: pipeline, agent, retrieval, or none of the above? Sep 2026 · 8 min read ### 03 Build How do we get from a working notebook to a deployable service? Sep 2026 · 6 min read ### 04 Evaluate How do we know it works, and how do we keep knowing after every change? Sep 2026 · 6 min read ### 05 Operate What breaks in production, and how do we see it before the user does? Sep 2026 · 8 min read ### 06 Cost What will this actually cost to run, and where does the spend hide? Sep 2026 · 6 min read ### AI Security Review What InfoSec, Privacy, Risk and Procurement ask before an AI feature ships, and the control and evidence that gets you through each gate. Sep 2026 · 23 min read ### The Agentic Delivery Playbook The operating model for running delivery with a fleet of coding agents: six control surfaces and 22 practices, measured by the metrics that catch review debt early. Sep 2026 · 26 min read ### The Eval Stack How to know an AI system works before it ships: the eval layers, plus the ways an LLM judge can quietly lie to you. Sep 2026 · 21 min read ### The Agent Tool Interface Agents fail at the tool interface more often than at the model. The design rules for building an interface that holds up in production. Sep 2026 · 26 min read ### Choosing an AI Delivery Partner How to interrogate any AI delivery partner, us included, across every gate that matters. It ends with the cases where we're the wrong call. Sep 2026 · 28 min read ### The AI Cost Model Where AI spend actually goes, the levers that move it, and the ways a cost estimate can already be wrong before you commit budget. Sep 2026 · 28 min read ### The AI Governance Layer The standing regime that has to hold a year after the security review passes. Who is accountable, and how to prove the system behaves. Sep 2026 · 30 min read ### The AI Release Path How to ship a change to an AI system and ramp it safely. Reverse it fast if it turns out to be wrong. Sep 2026 · 34 min read ### The Agent Adoption Playbook The pilot worked. How the rest of your engineers actually start using it, and the numbers that separate real adoption from seat activation. Sep 2026 · 33 min read ### MCP in Production [MCP](https://customlabs.io/glossary/model-context-protocol/) standardizes the wire format. Identity, the catalog and the trust boundary are still yours to build, plus a ten-question gate for a new server. Sep 2026 · 42 min read ### Context Engineering What occupies a [context window](https://customlabs.io/glossary/context-window/) at every step of a long run, and how to budget, compact, isolate and measure it before it fails. Sep 2026 · 37 min read ## 02 Architectures Six reference architectures for the systems we actually build, one level up from the patterns below. [All architectures →](https://customlabs.io/architectures/) ### Grounded answering over your own documents Answer questions from your own documents, with a citation, instead of from whatever the model learned during training. 7 min read ### Agentic workflow runner for a back-office queue An agent that works a queue of cases end to end, inside a bounded loop with a human checkpoint on anything it cannot cleanly undo. 7 min read ### Document extraction with a human review loop Turn inbound documents into structured records, with a person reviewing only the cases the model is actually unsure about. 6 min read ### An assistant inside a product you already ship An assistant embedded in a product you already ship, scoped to that signed-in user's own data and permissions. 7 min read ### Model-agnostic inference gateway One call site in front of every model provider, so a price change or an outage is a config change, not a rewrite. 6 min read ### A coding-agent delivery fleet A control plane for running many coding agents at once without them colliding on the same files or losing track of what any of it costs. 7 min read ## 03 Insights Dated articles on shipping applied AI. What breaks and how to design around it. [All insights →](https://customlabs.io/insights/) ### What Actually Moved in the EU AI Act's 2026 Deadline Change Regulation (EU) 2026/1744 pushed the AI Act's high-risk deadline to December 2027. What moved, what didn't, and why the old schedule still pays off. Aug 2026 · 9 min read ### Prompt Injection Is a Data Problem: A Threat Model You Can Ship Against [Prompt injection](https://customlabs.io/glossary/prompt-injection/) can't be filtered away: the model can't reliably tell instructions from data. Here's the actual threat model and the controls that hold up. Jul 2026 · 10 min read ### Why Your AI Agent Works in the Demo and Stalls in Production An agent that nails the demo stalls in production because reliability compounds across steps. Here's the math, the real failure modes, and how to ship anyway. Jul 2026 · 8 min read ### What AI Actually Costs in Production, by Workload A reproducible benchmark of cost per successful outcome across four common AI workloads, with every token assumption, price, and overhead multiplier shown. Jul 2026 · 9 min read ### Model-Agnostic by Design Models change under you every few months: price, quality, and capability. Here's why we never hardcode a single provider into a client's feature. Jul 2026 · 7 min read ### Your RAG Demo Lied to You Retrieval that looks flawless on ten clean PDFs falls apart on a real corpus. Here's why, and what evaluating retrieval quality actually requires. Jul 2026 · 7 min read ### What an AI Feature Actually Costs in Production Token costs that look trivial in a demo compound fast at scale. Here's how to make cost a first-class metric instead of a surprise on the invoice. Jun 2026 · 8 min read ### Evals Before You Ship: Why AI Features Need Tests Too Shipping an AI feature without an [eval suite](https://customlabs.io/glossary/eval-suite/) in CI means every prompt tweak is a guess. Here's what an eval suite actually needs to cover. Jun 2026 · 7 min read ### Build, Buy, or Skip: A Framework for AI Decisions A technically honest framework for deciding whether an AI initiative should be built in-house, bought off the shelf, or skipped entirely this year. May 2026 · 7 min read ### From Notebook to Production: What Actually Breaks The gap between a working AI demo and a production feature is auth, latency, cost and fallbacks. Here's how we close it without a rewrite. Apr 2026 · 7 min read ## 04 Comparisons Decision guides for the architecture calls that come up on every engagement. [All comparisons →](https://customlabs.io/compare/) ### Agents vs Pipelines: When Autonomy Is Worth the Reliability Cost Agentic autonomy and deterministic pipelines aren't a spectrum of ambition. They trade flexibility against reliability, and here's which your workflow needs. Jul 2026 · 7 min read ### Open-Weight Models vs Frontier APIs: The Real Cost/Control Tradeoff Self-hosting an open-weight model looks cheaper on paper than it behaves in practice. The real tradeoff against a frontier API, and when each one is right. Jul 2026 · 7 min read ### RAG vs Fine-Tuning: Which One Actually Solves Your Problem Retrieval and [fine-tuning](https://customlabs.io/glossary/fine-tuning/) solve different problems but get reached for interchangeably. How to tell which one your case needs, and why we default to retrieval. Jul 2026 · 8 min read ### Vector Database vs pgvector: Do You Actually Need a Dedicated Store A dedicated vector database is a bigger commitment than most retrieval workloads need on day one. Here's how to tell if pgvector already covers your case. Jul 2026 · 8 min read ## 05 Case studies Engagement write-ups: the problem, what we built, and the measured outcome. [All case studies →](https://customlabs.io/case-studies/) ### The AI-Native Target That Wasn't: A Two-Week Diligence A growth-equity firm's term sheet on an 'AI-native' SaaS target got repriced after a two-week check found $1.4M of hidden re-platform cost. Jul 2026 · 7 min read ### From Notebook to Production: An Extraction Model You Could Trust A healthtech's extraction model dazzled in a notebook, then hallucinated on real traffic. An eval harness cut [hallucinations](https://customlabs.io/glossary/hallucination/) from 12% to 3% in 8 weeks. Jun 2026 · 7 min read ### Cutting Inference Spend 40% Without Betting on One Vendor A Series B fintech's LLM spend outran revenue on one provider. Model-agnostic routing cut unit cost, vendor risk, and closed a data-residency gap. Jun 2026 · 7 min read ### Retrieval Pipeline That Actually Cut Support Load A SaaS support platform swapped a keyword search widget for a tenant-isolated retrieval pipeline, cutting escalations and response time with no new headcount. May 2026 · 6 min read ## 06 Patterns Named, opinionated patterns for building AI systems that survive production. [All patterns →](https://customlabs.io/patterns/) ### Intent router to specialists A cheap, fast classifier reads each request and routes it to one narrow specialist agent, instead of one god-agent that carries every tool. 6 min read ### Bounded agent loop An agent loop runs under an explicit budget: a maximum step count, a token ceiling, and a wall-clock limit. 6 min read ### Structure-aware chunking Chunk boundaries follow the document's own structure: headings, table rows, list items, section boundaries, instead of a fixed token count. 5 min read ### Retrieve-then-rerank A cheap, high-recall first pass pulls a wide candidate set of 50 to 100 documents likely to contain the right answer. 5 min read ### Change-data-capture ingest The ingest pipeline subscribes to the actual change events of the source system, such as a webhook, a CMS publish hook, or a database trigger. 5 min read ### Typed tool contract Every tool argument is defined by a strict JSON schema: enums for known value sets, validated patterns for IDs, required fields where genuinely needed. 5 min read ### Human checkpoint before irreversible actions Every tool the agent can call is scoped to the narrowest permission the task genuinely needs. 6 min read ### Golden-set gate in CI A fixed, human-labelled set of real cases runs automatically in CI on every prompt or model change. 5 min read ### Trace-first observability One trace ID follows a single request across every hop it takes: retrieval, every model call, every tool call. 5 min read ### Model cascade A cheap, fast model attempts every request first, and a validation signal decides whether that attempt is good enough to return. 5 min read ### Stable-prefix prompt caching The prompt is ordered with everything invariant across calls first: system instructions, tool definitions, few-shot examples. 5 min read ## 07 Failure modes Concrete ways production AI systems break, each with symptoms, root cause, and a fix. [All failure modes →](https://customlabs.io/failure-modes/) ### Stale index serves deleted content Your retrieval index was built once at ingest and never told the source changed. When a document is edited or deleted, nothing re-embeds the new version or tombstones the old chunk. The stale vector keeps scoring well and keeps getting served with confidence, giving the reader no signal that it is out of date. 4 min read ### Chunk boundary splits the answer The answer exists in the source, but a fixed-size chunker cut it in half at ingest time: a table row split from its header, a procedure split from its trigger condition. Each half scores weakly on its own, the ranker drops both, and retrieval reports nothing when the document plainly contains the answer. 4 min read ### Similarity is not relevance Cosine similarity rewards topical resemblance, not correctness. It can rank a document about the wrong product, the wrong date, or the negated version of a claim above the one that actually answers the query, because embeddings represent "about the same thing" far more reliably than they represent identifiers, negation, or numbers. 4 min read ### Unbounded agent loop The agent has no step budget, no token budget, and no way to recognize it is stuck. A failing tool call stays in its context and keeps looking like a reasonable next thing to try, so it keeps trying variations of the same failed approach until something external (a timeout, a bill, a human) stops it. 4 min read ### Tool argument hallucination Loose tool schemas (free-form string IDs, everything optional) give the model room to fill a gap with something plausible-looking instead of something real. With no server-side validation catching the mismatch before execution, a confidently invented ID reaches a system that expects a real one. 4 min read ### Silent tool failure The tool returned HTTP 200 with an error message in the body, or an empty result set, and the agent read the absence of data as evidence rather than as a failure. Nothing in the response forced a distinction between "nothing matched" and "something broke." 4 min read ### Context overflow drops the task As the conversation grows, a naive truncation strategy drops the oldest messages to stay under the context window. The oldest messages are exactly where the system prompt and the original task state usually live, so the agent keeps running with no memory of what it was actually supposed to do. 4 min read ### Vibes-based prompt regression Without a labelled eval set, the change was graded against whatever two or three examples the author happened to have open. That is not a test. It's an anecdote. A regression anywhere outside that narrow, unrepresentative sample ships straight to production undetected. ### Judge prefers its own output A judge from the same model family as the generator tends to rate that family's output favorably. This is self-preference bias. A single vague rubric ('is this good?') collapses almost everything to a passing score, so the eval suite stops being able to tell a real regression from noise. 4 min read ### Retry-amplified spend The estimate priced the happy path: one clean call per outcome. Production reality includes retries on malformed or rate-limited calls, fallbacks to a larger model when the first attempt fails, and agent loops that make several calls per completed task. Every one of those multiplies calls per successful outcome without multiplying the original per-token estimate. 4 min read ### Prompt cache never hits A dynamic prefix (a timestamp, a per-user greeting, a reordered tool list, retrieved chunks placed before the static instructions) changes the start of the prompt on every call. Prompt caching only pays off when the shared prefix is byte-identical across requests, and one volatile token near the front is enough to bust the whole cache. 4 min read ### Injection via retrieved content Yes. Retrieved content arrives on the same channel as instructions, so a document, ticket, or webpage crafted (or compromised) to contain commands can have the model execute them with its real tool permissions. The system has no built-in way to tell 'instruction from us' apart from 'text we retrieved.' 4 min read ## 08 Tools & diagnostics Free, self-serve tools and fixed-scope diagnostics for scoping AI work before you commit budget. [All tools →](https://customlabs.io/tools/) ### AI Cost Calculator What AI actually costs to run in production — not the sticker price. ### AI Readiness Scorecard A fast, honest read on whether your data, infra, and process are ready to ship AI. ### Architecture Picker Which system shape you should actually build, from seven questions about the problem. ### AI Readiness / Ship Audit A fixed-scope, fixed-fee read on whether you’re actually ready to ship AI. ### AI Technical Diligence An investor-grade read on a target’s AI claims, before you commit capital. ## 09 Topics Guides, architectures, comparisons, insights, case studies, patterns, failure modes and tools grouped by the six themes that recur across engagements. [All topics →](https://customlabs.io/topics/) - [Retrieval & RAG](https://customlabs.io/topics/retrieval-rag/) Topics - [AI Cost & Efficiency](https://customlabs.io/topics/ai-cost-efficiency/) Topics - [Evals & Observability](https://customlabs.io/topics/evals-observability/) Topics - [Shipping to Production](https://customlabs.io/topics/shipping-to-production/) Topics - [Strategy & Architecture](https://customlabs.io/topics/strategy-architecture/) Topics - [Security & Governance](https://customlabs.io/topics/ai-security-governance/) Topics ## 10 Glossary Plain-English definitions for the AI engineering vocabulary that matters. [All glossary terms →](https://customlabs.io/glossary/) - [Retrieval-Augmented Generation (RAG)](https://customlabs.io/glossary/retrieval-augmented-generation/) Glossary - [Embeddings](https://customlabs.io/glossary/embeddings/) Glossary - [Context Window](https://customlabs.io/glossary/context-window/) Glossary - [Token](https://customlabs.io/glossary/token/) Glossary - [Model-Agnostic Architecture](https://customlabs.io/glossary/model-agnostic-architecture/) Glossary - [Fine-Tuning (vs RAG)](https://customlabs.io/glossary/fine-tuning/) Glossary - [Vector Search (Semantic Search)](https://customlabs.io/glossary/vector-search/) Glossary - [Chunking](https://customlabs.io/glossary/chunking/) Glossary - [Agent (Agentic AI)](https://customlabs.io/glossary/agent/) Glossary - [Tool Calling](https://customlabs.io/glossary/tool-calling/) Glossary - [Model Context Protocol (MCP)](https://customlabs.io/glossary/model-context-protocol/) Glossary - [Context Engineering](https://customlabs.io/glossary/context-engineering/) Glossary - [Tool Contract](https://customlabs.io/glossary/tool-contract/) Glossary - [Idempotency](https://customlabs.io/glossary/idempotency/) Glossary - [Structured Output](https://customlabs.io/glossary/structured-output/) Glossary - [Guardrails](https://customlabs.io/glossary/guardrails/) Glossary - [Eval Suite (Evals)](https://customlabs.io/glossary/eval-suite/) Glossary - [LLM-as-Judge](https://customlabs.io/glossary/llm-as-judge/) Glossary - [Observability](https://customlabs.io/glossary/observability/) Glossary - [Hallucination](https://customlabs.io/glossary/hallucination/) Glossary - [Prompt Injection](https://customlabs.io/glossary/prompt-injection/) Glossary - [Inference Cost](https://customlabs.io/glossary/inference-cost/) Glossary - [Notebook-to-Production](https://customlabs.io/glossary/notebook-to-production/) Glossary - [Data Processing Agreement (DPA)](https://customlabs.io/glossary/data-processing-agreement/) Glossary - [Data Residency](https://customlabs.io/glossary/data-residency/) Glossary - [Red Teaming](https://customlabs.io/glossary/red-teaming/) Glossary - [AI Register](https://customlabs.io/glossary/ai-register/) Glossary - [High-Risk AI System](https://customlabs.io/glossary/high-risk-ai-system/) Glossary - [Human Oversight](https://customlabs.io/glossary/human-oversight/) Glossary - [System Card](https://customlabs.io/glossary/system-card/) Glossary - [Canary Release](https://customlabs.io/glossary/canary-release/) Glossary - [Output Drift](https://customlabs.io/glossary/output-drift/) Glossary - [Model Deprecation](https://customlabs.io/glossary/model-deprecation/) Glossary - [Shadow Deployment](https://customlabs.io/glossary/shadow-deployment/) Glossary - [Golden Path](https://customlabs.io/glossary/golden-path/) Glossary - [Shadow AI](https://customlabs.io/glossary/shadow-ai/) Glossary - [Acceptance Rate](https://customlabs.io/glossary/acceptance-rate/) Glossary - [Tool Poisoning](https://customlabs.io/glossary/tool-poisoning/) Glossary - [MCP Gateway](https://customlabs.io/glossary/mcp-gateway/) Glossary - [Confused Deputy](https://customlabs.io/glossary/confused-deputy/) Glossary - [Context Rot](https://customlabs.io/glossary/context-rot/) Glossary - [Compaction](https://customlabs.io/glossary/compaction/) Glossary - [Context Isolation](https://customlabs.io/glossary/context-isolation/) Glossary ## 11 The studio Who we are and how an engagement runs. - [Services](https://customlabs.io/services/) The studio - [AI Integrations](https://customlabs.io/services/ai-integrations/) The studio - [Custom Development](https://customlabs.io/services/custom-development/) The studio - [Strategy & Architecture](https://customlabs.io/services/strategy-architecture/) The studio - [Readiness & Diligence](https://customlabs.io/services/readiness-diligence/) The studio - [Products](https://customlabs.io/products/) The studio - [CodeHerder](https://customlabs.io/products/codeherder/) The studio - [CostMon](https://customlabs.io/products/costmon/) The studio - [FreeTier](https://customlabs.io/products/freetier/) The studio - [GreatAPIs](https://customlabs.io/products/greatapis/) The studio - [Beemy](https://customlabs.io/products/beemy/) The studio - [CustomHosted](https://customlabs.io/products/customhosted/) The studio - [Reserver](https://customlabs.io/products/reserver/) The studio - [txtfetch](https://customlabs.io/products/txtfetch/) The studio - [BizBinder](https://customlabs.io/products/bizbinder/) The studio - [The Delivery Record](https://customlabs.io/delivery-record/) The studio - [Capabilities](https://customlabs.io/capabilities/) The studio - [Process](https://customlabs.io/process/) The studio - [Pricing](https://customlabs.io/pricing/) The studio - [About](https://customlabs.io/about/) The studio - [FAQ](https://customlabs.io/faq/) The studio - [Accessibility](https://customlabs.io/accessibility/) The studio - [Contact](https://customlabs.io/contact/) The studio No entries match that combination. Clear a filter to see more.