Word2Vec,
explained.

Early NLP treated each word as a one-hot vector — a giant list of zeros with a single 1 marking that word's slot. In that scheme every word is exactly as far from every other; "Paris" and "France" are as unrelated as "Paris" and "banana." All the meaning is thrown away.

What you really want is a representation where similar words sit near each other — where the geometry itself carries meaning. That's a dense embedding: a few hundred real numbers per word instead of a lonely 1 in a sea of zeros.

Word2Vec's foundation is an old linguistic idea, the distributional hypothesis: words that show up in similar contexts tend to mean similar things. "Coffee" and "tea" appear around "drink," "cup," "hot," "morning" — so if you learn to predict context, words with similar contexts are pushed toward similar vectors.

The genius was making this cheap. Word2Vec is a tiny, shallow network — no deep stacks — trained on billions of words fast enough to actually run at scale.

The paper offers two mirror-image training games. CBOW (Continuous Bag of Words) hides the middle word and predicts it from the words around it. Skip-gram flips it: given the middle word, predict the words likely to surround it.

Nobody actually cares about these predictions. They're a pretext. The real prize is a by-product: to predict context well, the network is forced to arrange word vectors so that similar words cluster — and those vectors are what you keep. CBOW is faster; skip-gram shines on rare words and huge corpora.

The result that made the field gasp: the vectors captured relationships as consistent directions. The step from "man" to "woman" is roughly the same vector as "king" to "queen" — a gender direction. So you can literally compute:

"Paris − France + Italy" lands near "Rome." These analogies weren't programmed in — they emerged from nothing but learning to predict nearby words. Meaning had genuine mathematical structure.

Word2Vec was fast, simple, and stunningly good. It set records on word-analogy and similarity benchmarks, trained on billions of words on modest hardware, and produced vectors you could just download and plug into any NLP system. It turned "represent words as vectors" from a research curiosity into standard practice almost overnight.

Every embedding model owes Word2Vec its core premise: meaning can be a vector, and similarity can be distance. Modern systems use contextual embeddings from Transformers — where a word's vector shifts with its sentence, so "bank" by a river differs from "bank" with money — rather than one fixed vector per word. But the leap Word2Vec made is the one everything stands on.

When you build semantic search, a recommender, or a RAG retriever, you're placing things on a map of meaning and measuring nearness. That map was first drawn here.