RAG & Retrieval
All 10 cards in this deck
Chunking, hybrid search, reranking, grounding — the retrieval half of every AI product. Answer each question before you open it — then run the review above, which schedules every card with spaced repetition so it comes back right before you would forget it.
Why RAG instead of fine-tuning for company knowledge?
The index updates in seconds without retraining, answers can cite sources, and access control stays enforceable. Weights bake knowledge in stale.
The chunking tradeoff in one line?
Too small loses the context needed to understand the chunk; too large blurs retrieval and floods the prompt with irrelevant text.
Where does BM25 beat embeddings?
Exact strings: identifiers, error codes, names, SKUs — tokens embeddings blur. That’s why production retrieval is hybrid.
What is hybrid search?
Lexical (BM25) and semantic (vector) retrieval run in parallel and fused — usually by reciprocal rank fusion — covering each other’s blind spots.
Why rerank instead of retrieving more?
A cross-encoder reads query and document together — far more accurate than comparing embeddings, only affordable on the top ~100 candidates.
What is HNSW?
A layered proximity-graph index for approximate nearest-neighbor search — near-log-time lookups instead of scanning every vector, at some memory cost.
What is grounding?
Constraining answers to supplied sources — cite them, say “not found” otherwise, verify attribution. The production defense against hallucination.
What is the “lost in the middle” problem for RAG?
Models recall content at the start and end of the context best — retrieved passages buried mid-prompt get used least. Position matters.
Two-stage retrieval means…?
Stage 1: fast recall over millions (BM25 + vectors). Stage 2: slow precision over the shortlist (cross-encoder rerank).
How do you know your RAG is any good?
Retrieval evals on a golden set: did the right chunks surface (recall@k), and did the answer stay grounded in them — measured, not vibes.