Embeddings vs keyword search: meaning or exact terms?

Keyword search (classically BM25) matches on lexical term overlap, weighted by term frequency and document length — it finds documents that literally contain your words. Embedding search converts both query and documents into vectors and matches on geometric closeness — it finds documents about the same meaning, even if they share zero words with the query.

Hybrid search — running both a keyword index and an embedding index, then fusing the two ranked lists (commonly with reciprocal rank fusion) — is the modern default in production RAG systems, precisely because the two failure modes don't overlap. Embeddings recover queries with zero lexical overlap; keyword search recovers exact terms embeddings smooth over. Fusing them costs little beyond running both indexes, and closes each one's blind spot with the other.

The mechanical reason embeddings can lose to keyword search on exact terms isn't a training bug — it's what the representation is optimized for. An embedding model is trained to place semantically similar text near each other in vector space, which means it learns to treat near-synonyms and paraphrases as "close" on purpose. That same pressure that makes it good at meaning also smooths over the fine-grained token-level differences that make an exact-match query exact: v2.3.1 and v2.3.2 are extremely close in embedding space (they're nearly the same string, in nearly the same context), which is exactly wrong when the user needed the ONE document about the specific version with the bug, not "a document about roughly this version."

BM25 has no such smoothing — it's counting literal token overlap with frequency weighting, so it is, structurally, incapable of confusing "v2.3.1" with "v2.3.2" the way a vector-similarity match can. This is why the two methods aren't just "different," they fail in genuinely complementary ways: embeddings' strength (generalizing past exact wording) is mechanically the same property that causes its exact-match weakness.

Users of a developer tool's docs search bar type two very different kinds of queries. Natural questions like "why is my build slow" need semantic understanding — the right doc might be titled "Optimizing Compile Times" and share almost no words with the query, a case only embeddings handle well. But the same search bar also gets queries like the exact error string ERR_CONNECTION_RESET or a specific flag name, where the user needs the ONE doc that contains that exact token, and a pure embedding search risks surfacing several plausible-sounding-but-wrong pages about "connection issues" generally instead.

A hybrid retriever — BM25 for lexical precision, embeddings for semantic recall, fused into one ranked list — handles both query shapes from the same search bar, which is why it's the standard architecture rather than a compromise: neither index alone covers the actual distribution of real queries a documentation search bar receives.