Reranker
A cross-encoder model that re-scores retrieval candidates for higher precision.
Simple explanation
Vector search is fast but imprecise. A reranker reads the query and each candidate document together and scores relevance more accurately. Retrieve top-20 with vector search, rerank to top-5.
Engineering explanation
Rerankers use cross-encoders (e.g., Cohere Rerank, sentence-transformers CrossEncoder) that jointly encode query+document, producing a relevance score. This is more accurate than bi-encoder cosine similarity but too slow to run over the whole index. Two-stage retrieval: fast ANN for candidates, slow cross-encoder for the shortlist. Typical precision gain: 15-30%.
Example
Vector search returns 20 chunks → Cohere Rerank scores each against the query → top 5 are more relevant than the original top 5.
When to use it
Any production RAG where precision matters. Especially when vector-only search returns semantically-similar-but-irrelevant chunks.
Common misconception
Reranking is NOT optional in production RAG. Without it, top-k by cosine similarity is often wrong. Always rerank.