How do you get both broad recall and precise ranking from retrieval?
A cheap first pass pulls 50 to 100 candidate documents. A cross-encoder reranker then re-sorts them and applies a relevance floor. The first pass is usually vector search, optionally fused with keyword search. The cross-encoder scores the query and each candidate together. The floor lets the system return nothing when no candidate fits.
Also known as two-stage retrieval, cross-encoder rerank, candidate-then-rank
What problem does this solve?#
A single-stage vector searchVector search finds the nearest matches to a query by comparing embeddings, not exact keywords. must find the right document among thousands. It must also rank that document above near-misses. It does both with one similarity score, computed for each document independently of the others. Bi-encoder similarity finds documents on the same topic. It is much weaker on the detail that decides correctness, such as an identifier or a negation. A topically similar but wrong document can outrank the correct one.
Retrieval that always returns its top-k has no floor. It treats "nothing in the corpus answers this" the same as "here is the answer." Both come back as a ranked list. A downstream model reading that list has no signal to say "I do not know". It answers with confidence from the best available match, even when that match is wrong.
How does it work?#
Stage one casts a wide net cheaply with vector search. Optionally, it fuses BM25 keyword search to catch identifier and exact-phrase cases that vectors miss. It pulls a candidate set of 50-100 documents and favors recall over precision. False positives are fine here, because stage two filters them.
Stage two scores the query jointly against each candidate with a cross-encoder. This costs more per comparison than bi-encoder similarity. It is far more accurate, because it attends to both texts together. Two independently computed vectors cannot do that. The cost stays affordable because the cross-encoder runs only on the narrowed candidate set.
A relevance floor on the reranked scores decides whether anything qualifies as an answer. If the top reranked result falls under the floor, the system returns "no confident match." The floor is a design decision. The team usually validates it against a labelled eval set. The system treats returning nothing as correct behavior.
What are the moving parts?#
- Retrieve a wide candidate set
Pull 50-100 candidates with a cheap, high-recall method. Favor recall over precision at this stage.
- Fuse in keyword retrieval where useful
Add BM25 keyword search next to vector search for identifier and exact-phrase queries.
- Score jointly with a cross-encoder reranker
Re-rank the narrowed candidate set with a model that reads the query and each candidate together.
- Apply a relevance floor
Use a labelled eval set to pick the reranked score below which nothing qualifies as an answer.
- Return "no confident match" below the floor
Treat returning nothing as correct behavior when no result clears the floor.
- Feed only floor-cleared results downstream
Generation sees only the reranked results that pass the floor. It never sees the raw first-stage candidate set.
When does it fit, and when doesn't it?#
Use it when
- Retrieval quality complaints trace back to a topically similar but wrong document that outranks the correct one
- The corpus is large enough that single-stage similarity search has precision problems as well as recall problems
- Some queries have no answer in the corpus, and a confident wrong answer is worse than an admission of that
- The latency budget and compute budget allow a second scoring pass on a narrowed candidate set
Don't use it when
- The corpus is small and single-stage retrieval already has near-perfect precision. A reranker adds latency for no gain
- The latency budget cannot absorb a second model call in the retrieval path, as in hard real-time systems
- No labelled eval set exists to calibrate the relevance floor. An uncalibrated floor blocks good answers or lets bad ones through, and the team cannot tell which
What does it cost?#
Better precision than single-stage similarity search, especially on near-miss and identifier-heavy queries
The cross-encoder adds a second model call to every request. That call adds latency and compute cost
A relevance floor lets the system say "no answer" and stop guessing
Calibrating the floor takes a labelled eval set. A wrong floor suppresses good answers or lets bad ones through
Keyword fusion recovers the exact-match and identifier queries that pure vector search misses
Rank fusion across two scoring methods needs its own tuning
Reranking only the narrowed candidate set keeps the expensive model affordable
The first stage caps recall. The reranker cannot recover a document the first stage never surfaced
How do you know it's working?#
- Precision@k on the final reranked result set, with the floor applied, against a labelled eval set
- Rate of "no confident match" responses, watched for both under-triggering and over-triggering
- Added latency from the rerank stage (p50/p95)
- Recall@k of the first-stage candidate set, because it caps everything downstream
What failure modes does this prevent?#
Sources
- arXiv - Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
The paper that named retrieval-augmented generation as a technique. Retrieved 2026-08-24.
- arXiv - Passage Re-ranking with BERT
The paper that established cross-encoder scoring as the second-stage reranking method our retrieve-then-rerank pattern applies. Retrieved 2026-09-17.
A Ship Audit checks which of these patterns your system needs. It ranks them by what is most likely to break first.