One-Hot Encoding
Representing a category as a vector of all zeros with a single one — the raw form before dense embeddings.
One-hot encoding represents a categorical value (a word, a class) as a vector the size of the vocabulary — all zeros except a single one at that item’s index. It is the raw input form before a model maps it to a dense embedding.
Worked example: in a 50,000-word vocabulary the token “cat” is a 50,000-long vector with a 1 at cat’s index; multiplying it by the embedding matrix simply selects cat’s embedding row. Gotcha: one-hot vectors are huge, sparse, and carry no notion of similarity — every pair is equally distant — which is exactly the problem embeddings solve.