Chunking
Splitting documents into retrievable pieces — the decision that quietly makes or breaks RAG quality.
Chunking cuts documents into units for embedding and retrieval. Too small and chunks lose the context needed to be understood; too large and retrieval gets fuzzy while irrelevant text floods the prompt. Practical systems chunk on structure (headings, paragraphs, code blocks) with modest overlap, attach parent-document context, and tune size against retrieval evals — not intuition.
Worked example: splitting a document into ~500-token chunks lets retrieval return just the relevant passage; too large and one vector blurs multiple topics, too small and a chunk loses the context needed to answer. Gotcha: naive fixed-size splits cut sentences and separate a claim from its qualifier — overlap or structure-aware splitting preserves meaning that character-count chunking destroys.