Retrieval-Augmented Generation (RAG)

RAG retrieves passages relevant to a query (usually by embedding similarity) and adds them to the context before generation, grounding answers in your data and reducing hallucination. Quality hinges on chunking, embedding, and how the top results are selected and ranked.

Worked flow: embed the question → find the top-k nearest chunks by cosine similarity → paste them into the prompt with a “answer only from these sources” instruction → generate with citations. Gotcha: retrieval quality caps answer quality — if the right passage is not in the top-k, the model answers confidently from the wrong context, and no bigger model fixes it. Most “RAG is bad” problems are retrieval problems (chunking, embedding, ranking), not generation problems.

Worked example: retrieval-augmented generation: before answering, fetch relevant chunks from a knowledge base (via embedding search) and put them in the prompt, so the model answers from your data instead of only its trained-in memory. Gotcha: RAG quality is bottlenecked by retrieval — if the right chunk is not in the top-k the model cannot use it (and may hallucinate to fill the gap); chunking, embedding quality, and reranking matter more than the generation model, and RAG fixes stale/private knowledge, not reasoning.