Retrieval

Hybrid Search

Combine dense vector search with sparse keyword (BM25) search and fuse the results for more robust retrieval.

Query

Dense (Vector)

Sparse (BM25)

RRF Fusion

Reranker

LLM

clientservicemodeldatabaseagentexternal

Explanation

The query is searched both as an embedding (semantic) and as keywords (exact match / IDs / names). Results are fused with Reciprocal Rank Fusion, then reranked by a cross-encoder before being passed to the LLM.

Components

Vector index (pgvector)BM25 index (Postgres tsvector / OpenSearch)RRF fusionReranker

When to use

RAG over mixed content (prose + code + identifiers)
When users search by name or ID
Production search where recall matters

When NOT to use

Pure semantic Q&A over prose
Latency-critical < 200ms

Failure modes

Index drift between dense and sparse
Fusion weights wrong for the domain
BM25 not tokenised correctly for the language

Production checklist

Both indexes rebuilt atomically
RRF weights tuned on eval set
Reranker on top
Recall and precision measured per branch