Retrieval-augmented generationRetrieval-Augmented Generation (RAG) retrieves relevant passages at query time and feeds them into an LLM's context. looks simple in a demo and breaks on real corpora — messy PDFs, thin chunking, and stale embeddingsEmbeddings are numeric vectors that place similar content close together in vector space. all masquerade as model problems. This is our work on making retrieval actually hold up in production.
Start with Context EngineeringContext engineering decides what occupies a model's context window at every step, and in what order. if you're building retrieval or RAG. It's for engineers who need their system to reason over real documents, not a clean demo corpus.
What occupies a context windowThe context window is the maximum text, measured in tokens, a model can consider at once. at every step of a long run, and how to budget, compact, isolate and measure it before it fails.
37 min read Read →Answer questions from your own documents, with a citation, instead of from whatever the model learned during training.
7 min read Read →Turn inbound documents into structured records, with a person reviewing only the cases the model is actually unsure about.
6 min read Read →Retrieval and fine-tuningFine-tuning trains a model's weights on your own examples, changing its behavior directly. solve different problems but get reached for interchangeably. How to tell which one your case needs, and why we default to retrieval.
8 min read Read →A dedicated vector database is a bigger commitment than most retrieval workloads need on day one. Here's how to tell if pgvector already covers your case.
8 min read Read →Retrieval that looks flawless on ten clean PDFs falls apart on a real corpus. Here's why, and what evaluating retrieval quality actually requires.
7 min read Read →A SaaS support platform swapped a keyword search widget for a tenant-isolated retrieval pipeline, cutting escalations and response time with no new headcount.
6 min read Read →Chunk boundaries follow the document's own structure: headings, table rows, list items, section boundaries, instead of a fixed token count. Each chunk carries its parent heading or identifying context in its own body. A table row is never separated from its header, and a step is never separated from the procedure it belongs to. The chunk that gets embedded is always a complete unit of meaning, not an arbitrary slice.
Read →A cheap, high-recall first pass pulls a wide candidate set of 50 to 100 documents likely to contain the right answer. It's usually vector search, optionally fused with keyword search. A precision reranker, usually a cross-encoder that scores the query and each candidate jointly, then re-sorts that set. It applies a relevance floor, allowed to return nothing rather than force a weak match to the top.
Read →The ingest pipeline subscribes to the actual change events of the source system, such as a webhook, a CMS publish hook, or a database trigger. That replaces re-crawling on a fixed schedule. An edit triggers a re-embed within minutes. A delete writes a tombstone that excludes the old chunks from retrieval immediately, before a full re-index even runs.
Read →Your retrieval index was built once at ingest and never told the source changed. When a document is edited or deleted, nothing re-embeds the new version or tombstones the old chunk. The stale vector keeps scoring well and keeps getting served with confidence, giving the reader no signal that it is out of date.
Read →The answer exists in the source, but a fixed-size chunker cut it in half at ingest time: a table row split from its header, a procedure split from its trigger condition. Each half scores weakly on its own, the ranker drops both, and retrieval reports nothing when the document plainly contains the answer.
Read →Cosine similarity rewards topical resemblance, not correctness. It can rank a document about the wrong product, the wrong date, or the negated version of a claim above the one that actually answers the query, because embeddings represent "about the same thing" far more reliably than they represent identifiers, negation, or numbers.
Read →Source: https://customlabs.io/topics/retrieval-rag/