Meaning,
as geometry.

An embedding is a vector — an ordered list of numbers — that stands in for a piece of content: a word, a sentence, a whole document, an image. The magic isn't the vector; it's the training. Embedding models are trained so that content with similar meaning gets nearby vectors. "dog" sits near "puppy"; "bank" (money) drifts away from "bank" (river). Meaning becomes position.

Once meaning is position, hard questions turn easy. "Which of these 10,000 documents is most relevant to my question?" becomes "which vector is closest to the query vector?" — a geometry problem with fast, exact answers. That single reframing is why embeddings underpin semantic search, clustering, deduplication, recommendation, and retrieval-augmented generation. The rest of this handbook is about the one measurement that makes "closest" precise.

The default way to compare two embeddings is cosine similarity — the cosine of the angle between them. It's the dot product of the two vectors divided by the product of their lengths:

Same direction → cosine 1 (most similar). Perpendicular → cosine 0 (unrelated). Opposite → cosine −1. It runs from −1 to 1.

The crucial property is that cosine measures direction, not magnitude. A long document and a short one about the same topic should count as similar even though one vector is much longer — dividing by the lengths strips magnitude away and leaves only the angle. That's usually exactly what you want for meaning: "aboutness" shouldn't depend on how many words you used. (When magnitude does carry signal, raw dot product or Euclidean distance may be preferred — but for semantic text, cosine is the safe default.)

Here's the trick that vector databases live on. Dividing by lengths on every comparison is wasteful when you're doing millions of them. So instead, L2-normalize every vector once — scale it to unit length — up front. After that, the denominator is always 1, and the dot product equals the cosine similarity:

Normalize once at index time; then ranking by cosine is just ranking by dot product — which is what fast maximum-inner-product search (and indexes like HNSW) are built to do.

This is why so much embedding code normalizes vectors and then never mentions cosine again — the cosine is baked in. It also makes distances behave nicely: on unit vectors, Euclidean distance and cosine similarity become monotonically related, so nearest-by-distance and most-similar-by-cosine agree.

Real embeddings aren't 2-D — they're hundreds to thousands of dimensions (384, 768, 1536, 3072 are common). More dimensions give the model more room to separate fine distinctions in meaning, which is why bigger embedding models often retrieve better. But dimensions aren't free: storage and search cost scale with them, and very high dimensions bring the curse of dimensionality — distances between random points concentrate, so "nearest" gets less discriminative unless the embedding genuinely uses those dimensions well.

The practical upshot is a trade-off you tune. Smaller vectors are cheaper to store and faster to search; larger ones can be more accurate. Techniques like quantization compress vectors to reclaim memory, and some modern models support Matryoshka embeddings you can truncate to a shorter prefix for a cheaper approximation. The geometry — cosine on normalized vectors — stays identical no matter the dimension; only the cost and the fidelity change.

Once you can measure similarity, a lot of products fall out of the same primitive:

In practice the pipeline is: pick an embedding model, embed and normalize your corpus, store the vectors in an index (with an ANN structure for scale), and at query time embed, normalize, and retrieve by dot product. The vector database handles the index; the embedding model determines quality. Both matter: a great index over bad embeddings retrieves the wrong things quickly.

Embeddings are simple to use and easy to misuse. A few traps worth naming: model mismatch — you must embed queries and documents with the same model (and often the same instruction/prefix); mixing models compares vectors from different spaces, which is meaningless. forgetting to normalize — if you rank by dot product but didn't normalize, longer documents get an unfair boost. domain drift — a general model may not separate your niche jargon well; sometimes a domain-tuned model or a late-interaction retriever helps.

And the deepest one: embeddings capture similarity, not truth or relevance-to-your-task. Two sentences can be semantically close yet one is right and one is wrong; a passage can be on-topic but unhelpful. That's why strong retrieval systems pair embeddings with a reranker and, for questions, a generator that reads the retrieved text. Embeddings get you the right neighborhood fast; the last mile of precision usually needs another step. Used with that awareness, though, this one idea — meaning as geometry — is the most reusable primitive in applied AI.