Advanced50 minProduction RAG

Reranking for Retrieval Precision

Add a cross-encoder reranker on top of vector search and measure precision improvement.

Scenario

Your vector search recall is good but precision is poor — too many irrelevant chunks in top-5. Add a reranker.

Objective

Retrieve top-20 with vector search, rerank to top-5, and measure precision@5 improvement.

Starter code

Implement the TODOs to complete the lab.

rerank.pypython

def rerank(query: str, candidates: list[dict], top_k: int = 5) -> list[dict]:
    """Rerank candidates by relevance to query using a cross-encoder.
    Return top_k sorted by relevance score.
    """
    # TODO — use Cohere rerank or sentence-transformers CrossEncoder
    pass

def evaluate_with_rerank(queries: list[dict]) -> dict:
    """Compare precision@5 with and without reranking."""
    # TODO
    pass

Solution hints

1Cross-encoder scores query-document pairs jointly
2Retrieve top-20 first, rerank to top-5
3Measure precision@5 before and after
4Expect 15-30% precision improvement

Validation steps

Your implementation should pass these checks.

Reranker returns top_k sorted by score
Precision@5 improves vs no-rerank baseline
Latency overhead is acceptable (< 200ms)
Handles empty candidate list gracefully

Run validation

This is a simulated validation environment. In production, this would run your code against the validation steps.