What is RAG and how does it work? Walk me through the full pipeline.
Think first
Think about: ingestion, chunking, embedding, retrieval, generation, citation. What are the production additions?
Strong answer
RAG (Retrieval Augmented Generation) grounds LLM responses in retrieved documents. The pipeline: (1) Ingestion — extract text from sources (PDF, HTML), clean, and add metadata. (2) Chunking — split into ~512-token units with overlap, section-aware. (3) Embedding — convert chunks to vectors. (4) Store in a vector database (pgvector) with metadata. (5) Query — embed the query, search for top-k similar chunks (optionally hybrid: vector + BM25). (6) Reranking — cross-encoder reranks top-20 to top-5 for precision. (7) Generation — inject top chunks as context, LLM generates answer with citations. Production adds: access-control filtering at the SQL level, caching, query rewriting, observability on retrieval scores, and evaluation (faithfulness, recall@k).
Weak-answer indicators
- Says 'RAG makes the model smarter' (it grounds, doesn't make smarter)
- Skips chunking or reranking
- Doesn't mention citations or refusal when context is insufficient
- Confuses RAG with fine-tuning
Likely follow-up questions
- How would you debug a RAG system with good retrieval but poor answers?
- When would you choose RAG over fine-tuning?
- How do you handle access control in a multi-tenant RAG system?