# What shape is the system: pipeline, agent, retrieval, or none of the above?

Source: https://customlabs.io/handbook/design/
Updated: 2026-09-12

Handbook / 02

# What shape is the system: pipeline, agent, retrieval, or none of the above?

Updated September 12, 2026 · First published July 29, 2026

·

8 min read

·

Design

Once "build" wins, the next decision is architectural: a fixed pipeline, an autonomous agent, a retrieval layer, or some combination. Each shape carries a different reliability profile before you write the first prompt.

Guide

## What actually happens at this stage

Design is where most of the long-term reliability of an AI system gets decided, and it happens before any model is called in anger. The central fork is agent versus pipeline: a pipeline is a fixed sequence of steps where the model fills in one or two of them, predictable and easy to test because the control flow doesn't change; an agent is a loop where the model decides what happens next, powerful for open-ended tasks and much harder to bound, test, and debug. Most systems that get called "an AI agent" in a pitch deck would be more reliable, cheaper, and easier to eval as a pipeline. Reach for a real agent only when the task's steps genuinely can't be known in advance.

If any part of the system needs to answer from your own data, retrieval enters the design, and it deserves the same deliberateness as the agent-vs-pipeline call. The default architecture is chunk, embed, retrieve by similarity, rerank, generate. Every one of those five verbs is a place a demo-passing system quietly breaks in production. [Chunking](https://customlabs.io/glossary/chunking/) that ignores document structure splits tables and procedures in half; [embeddings](https://customlabs.io/glossary/embeddings/) encode topical resemblance far better than exact identifiers, dates, or negation; a system that looks solid on ten demo queries can fail badly on the identifier-shaped queries real users actually ask.

The vector-database-versus-pgvector fork belongs here too, and it's smaller than it looks from the outside: if you already run Postgres and your corpus is in the low millions of vectors, pgvector removes an entire piece of infrastructure to operate for a real but bounded performance cost; a dedicated vector database earns its keep once scale, filtering complexity, or query latency requirements outgrow what an extension bolted onto your primary database can deliver.

Whichever shape you land on, design it to be model-agnostic from the start, not as a retrofit after the first provider price change or deprecation notice. That means a real abstraction layer between your application and any single model provider, even a thin one, so a workload can route to whichever model is cheapest or most capable for that specific call, and a provider outage or pricing change becomes a config change rather than a rewrite. Almost every team that skips this does so because it looks like premature abstraction on day one; almost every team that skips it retrofits it within a year, under worse conditions than they'd have chosen for themselves.

Whatever shape emerges (agent, pipeline, retrieval, or a combination) should come out of this stage as a small number of named, deliberate patterns, not an emergent pile of prompts. An intent router in front of specialist agents instead of one god-agent; a bounded loop with an explicit step and token budget instead of an open-ended one; typed [tool contracts](https://customlabs.io/glossary/tool-contract/) instead of free-form string arguments. We hit exactly this coordination problem operating fleets of coding agents ourselves. It's the reason CodeHerder exists, as a shared task registry and per-task cost ledger instead of a wall of uncoordinated terminal tabs. None of these are exotic; they're the standard shapes for the failure modes this stage is trying to design around before they happen, and adopting the two or three your system actually needs beats adopting all of them.

Mistake

## The common mistake

The most common mistake at this stage is reaching for an agent when a pipeline would do. Teams build an open-ended loop for a task whose steps were actually knowable in advance, then spend the next quarter building the [guardrails](https://customlabs.io/glossary/guardrails/) a pipeline would never have needed in the first place.

Done

## How do you know this stage is finished?

- You've explicitly chosen agent, pipeline, retrieval, or a combination, and can say why the alternative shapes were rejected.
- If retrieval is involved, chunking respects document structure and you've checked identifier and negation queries specifically.
- The vector-store decision (pgvector vs. a dedicated database) is sized to your actual scale, not defaulted to whichever is trendiest.
- There's a model-agnostic layer between your application and any single provider, even a thin one.
- The design maps to two or three named patterns your system actually needs, not every pattern in the library.

Sources

## What backs this up

### Insights

- [Model-Agnostic by Design](https://customlabs.io/insights/model-agnostic-by-design/) Why the abstraction layer belongs in the design, not in the retrofit after the first price change.

### Case studies

- [Retrieval Pipeline That Actually Cut Support Load](https://customlabs.io/case-studies/retrieval-pipeline-cut-support-load/) What a retrieval design that respects structure and identifiers actually buys you in production.

### Comparisons

- [Agents vs Pipelines: When Autonomy Is Worth the Reliability Cost](https://customlabs.io/compare/agents-vs-pipelines/) The central fork of this stage. Decide it on purpose, not by whichever shape a pitch deck defaulted to.
- [Vector Database vs pgvector: Do You Actually Need a Dedicated Store](https://customlabs.io/compare/vector-database-vs-pgvector/) Smaller than it looks: sized to your actual scale, not to whichever is trendiest.

### Patterns

- [Intent router to specialists](https://customlabs.io/patterns/intent-router-to-specialists/) The standard shape for "one agent that knows everything" before it becomes a reliability problem.
- [Bounded agent loop](https://customlabs.io/patterns/bounded-agent-loop/) The termination contract every agent design needs before it ships, not after it loops for 40 minutes.
- [Typed tool contract](https://customlabs.io/patterns/typed-tool-contract/) Tighter schemas at design time are what stop a model inventing a plausible-looking tool argument later.
- [Structure-aware chunking](https://customlabs.io/patterns/structure-aware-chunking/) The retrieval design decision most fixed-size chunkers get wrong before anyone notices.
- [Retrieve-then-rerank](https://customlabs.io/patterns/retrieve-then-rerank/) The two-stage shape that recovers precision pure [similarity search](https://customlabs.io/glossary/vector-search/) can't reach alone.

### Glossary

- [Agent (Agentic AI)](https://customlabs.io/glossary/agent/) What actually makes something "agentic" is a spectrum, not the on/off switch the agent-vs-pipeline fork implies.
- [Tool Calling](https://customlabs.io/glossary/tool-calling/) The primitive underneath every agent design in this stage.
- [Model Context Protocol (MCP)](https://customlabs.io/glossary/model-context-protocol/) The standard worth adopting early if the model-agnostic call above is genuine.
- [MCP Gateway](https://customlabs.io/glossary/mcp-gateway/) Where several of these design decisions (identity, logging, rate limits) get built once instead of duplicated per integration.
- [Tool Contract](https://customlabs.io/glossary/tool-contract/) The schema that stops a model inventing a tool argument. Tighten it at design time, not after an incident.
- [Context Engineering](https://customlabs.io/glossary/context-engineering/) What deciding the shape of the agent design above is actually called once tool defs and retrieved content enter the same window.
- [Chunking](https://customlabs.io/glossary/chunking/) The retrieval decision with the most leverage over whether the design actually works on real documents.
- [Embeddings](https://customlabs.io/glossary/embeddings/) What a retrieval design is actually built on, and where its blind spots come from.
- [Vector Search (Semantic Search)](https://customlabs.io/glossary/vector-search/) Why it's usually paired with metadata filters and a reranker rather than trusted alone.
- [Model-Agnostic Architecture](https://customlabs.io/glossary/model-agnostic-architecture/) The design property this stage's provider-independence argument is actually named after.

### Tools

- [Architecture Picker](https://customlabs.io/tools/architecture-picker/) Turn the architecture-hub self-diagnosis into a scored primary and bolt-on before you commit to one.

### More

- [CodeHerder](https://customlabs.io/products/codeherder/) The bounded-loop, task-registry coordination problem above, solved at the scale of a whole agent fleet.
- [The Agent Tool Interface](https://customlabs.io/tool-design/) The full pillar behind the tool contract and typed-tool-contract calls above: six interface surfaces, 24 named design rules, and five ways a tool interface fails.
- [Reference Architectures](https://customlabs.io/architectures/) The system-level hub this stage's shape decisions assemble into: six named architectures, each with the components, cost shape, and security posture that only make sense once the pieces are together.
- [Grounded answering over your own documents](https://customlabs.io/architectures/grounded-answering/) The retrieval design decisions above, assembled into the full system an "answer from our own docs" request actually needs.
- [Agentic workflow runner for a back-office queue](https://customlabs.io/architectures/agentic-workflow-runner/) The agent-vs-pipeline and bounded-loop calls above, assembled into the shape a back-office queue automation request actually needs.
- [An assistant inside a product you already ship](https://customlabs.io/architectures/in-product-copilot/) The model-agnostic and tool-scoping design decisions above, applied to an assistant embedded inside a product you already ship.
- [Model-agnostic inference gateway](https://customlabs.io/architectures/model-gateway/) This stage's model-agnostic-by-design argument, assembled into the gateway that actually makes a provider swap a config change.

Questions

## Questions on this stage

What comes up before and during design.

01 Do we need an agent, or is a pipeline enough? +

Default to a pipeline. Reach for an agent only when the task's steps genuinely can't be known in advance. Most systems pitched as "agents" are pipelines with one or two model-filled steps, and would be more reliable built that way on purpose.

02 pgvector or a dedicated vector database? +

If you already run Postgres and your corpus is in the low millions of vectors, start with pgvector. It removes a whole piece of infrastructure to operate. Move to a dedicated vector database once scale, filtering, or latency requirements outgrow it.

03 Why does model-agnostic architecture matter this early? +

Because it's cheap to build in from the start and expensive to retrofit. Almost every team that skips it ends up doing the retrofit within a year, usually right after a price change or deprecation they didn't choose the timing of.

Not sure where you are in this?

A Ship Audit checks your actual system against every stage of this handbook and hands back a written, prioritized plan.

[Book a Ship Audit →](https://customlabs.io/diagnostic/ship-audit/)