Retrieval-augmented generationRetrieval-Augmented Generation (RAG) retrieves relevant passages at query time and feeds them into an LLM's context. looks simple in a demo and breaks on real corpora — messy PDFs, thin chunking, and stale embeddingsEmbeddings are numeric vectors that place similar content close together in vector space. all masquerade as model problems. This is our work on making retrieval actually hold up in production.
Start with Context EngineeringContext engineering decides what occupies a model's context window at every step, and in what order. if you're building retrieval or RAG. It's for engineers who need their system to reason over real documents, not a clean demo corpus.
What occupies a context windowThe context window is the maximum text, measured in tokens, a model can consider at once. at every step of a long run, and how to budget, compact, isolate and measure it before it fails.
38 min read Read →Answer questions from your own documents and cite the source. The model does not answer from its training data.
6 min read Read →Turn inbound documents into structured records. A person reviews only the cases where the model is unsure.
6 min read Read →Retrieval and fine-tuningFine-tuning trains a model's weights on your own examples, changing its behavior directly. solve different problems but get reached for interchangeably. How to tell which one your case needs, and why we default to retrieval.
7 min read Read →A dedicated vector database is a bigger commitment than most retrieval workloads need on day one. This piece shows how to tell if pgvector covers your case.
8 min read Read →Retrieval that looks flawless on ten clean PDFs falls apart on a real corpus. We explain why, and what a retrieval eval requires.
7 min read Read →A SaaS support platform swapped a keyword search widget for a tenant-isolated retrieval pipeline, cutting escalations and response time with no new headcount.
6 min read Read →Chunk boundaries follow the document's own structure, such as headings and table rows. Each chunk carries its parent heading in its own body. A table row stays with its header. A step stays with its procedure. Each embedded chunk is a complete unit of meaning.
Read →A cheap first pass pulls 50 to 100 candidate documents. A cross-encoder reranker then re-sorts them and applies a relevance floor. The first pass is usually vector search, optionally fused with keyword search. The cross-encoder scores the query and each candidate together. The floor lets the system return nothing when no candidate fits.
Read →The ingest pipeline subscribes to change events from the source system. This replaces re-crawling on a fixed schedule. The events come from a webhook or a CMS publish hook. An edit triggers a re-embed within minutes. A delete writes a tombstone that excludes the old chunks from retrieval at once, before a full re-index runs.
Read →Your retrieval index was built once at ingest and never told the source changed. When a document is edited or deleted, nothing re-embeds the new version or tombstones the old chunk. The stale vector keeps scoring well and keeps getting served with confidence, giving the reader no signal that it is out of date.
Read →The answer exists in the source, but a fixed-size chunker cut it in half at ingest time. A table row lost its header. A procedure lost its trigger condition. Each half scores weakly on its own, the ranker drops both, and retrieval reports nothing when the document plainly contains the answer.
Read →Cosine similarity rewards topical resemblance, not correctness. It can rank a document about the wrong product or the wrong date above the one that answers the query. It can do the same for the negated version of a claim. Embeddings capture topical similarity far more reliably than they capture identifiers, negation, or numbers.
Read →Source: https://customlabs.io/topics/retrieval-rag/