Chunking
Splitting long documents into smaller units for embedding and retrieval.
Simple explanation
LLMs and embedding models have input limits. Chunking breaks documents into smaller pieces so each can be embedded and retrieved independently. Good chunking preserves context; bad chunking splits concepts across boundaries.
Engineering explanation
Chunking strategies: fixed-size (simple, loses structure), recursive (splits on headers then size), semantic (groups by meaning), section-aware (uses document structure). Production default: 512 tokens with 64 overlap. Chunk size trades off precision (small) vs recall (large). Overlap bridges concepts across boundaries.
Example
A 50-page policy document → 200 chunks of ~512 tokens each, each with metadata (source, section, permission_group).
When to use it
Always, in RAG. The chunk is the unit of retrieval — it defines what 'similar' means.
Common misconception
Chunking is NOT just 'splitting text'. The strategy (section-aware, semantic) matters more than the size. Poor chunking is the #1 cause of bad RAG.