Cosine Similarity

Start from brute force

The tempting shortcut is to score similarity with the raw dot product, sum(x * y for x, y in zip(a, b)). It is one pass, O(d) time and O(1) space, and it does reward alignment, but it also grows with length: [2, 2]·[2, 2] = 8 outscores [1, 0]·[1, 0] = 1, although both pairs point in exactly the same direction. Euclidean distance fails the other way: [1, 1] and [2, 2] are about 1.41 apart even though their directions are identical. The question is about direction only, so the score has to divide out both lengths.

The approach

Cosine is the dot product of the two unit vectors, so divide out both lengths. The reference cosine_similarity computes three sums: dot = sum(x * y for x, y in zip(a, b)), then norm_a and norm_b as math.sqrt of each vector’s sum of squares, and returns dot / (norm_a * norm_b). A scale factor on either vector multiplies the numerator and the denominator alike, so it cancels, which is the length-independence the problem asks for. The Cauchy–Schwarz inequality, |a·b| ≤ ‖a‖‖b‖, is why the answer lands in [-1, 1].

The TypeScript cosineSimilarity fuses the three sums into one loop with running totals dot, na and nb, and takes both square roots only at the end. Either way the work is three multiply-adds per dimension. The constraints rule out an all-zero vector; without that promise you need an explicit guard, because the norm product is 0 and the division fails.

Walk through an example

For the first example, a = b = [1, 2, 3], the running sums grow together and the division returns exactly 1.0.

Complexity

Time O(d) for d dimensions: three multiply-adds per index, whether done as three passes (Python) or one loop (TypeScript). Space O(1): three running sums. The raw dot product is also O(d) time and O(1) space, so normalizing costs only two extra sums and two square roots. Comparing one query against N stored vectors is O(N·d) either way.

Edge cases to test

• a = [1, 1], b = [2, 2] → 0.9999999999999998, not exactly 1.0, because sqrt(2) * sqrt(8) rounds to 4.000000000000001. Compare with a tolerance, as the tests do, never with ==.
• a = b = [1, 5] → 1.0000000000000002: rounding can also land just above 1, so clamp to [-1, 1] before passing a score to acos.
• a = [0, 0], b = [1, 1]: the norm product is 0, so Python raises ZeroDivisionError and TypeScript returns NaN. The constraints exclude it; production code should return 0 or raise a clear error.
• a = [1, 2, 3], b = [1, 2] → Python returns 0.5976…, because zip stops at the shorter list while norm_a still uses all of a; TypeScript returns NaN. Check the lengths first, or use zip(a, b, strict=True) in Python 3.10+.
• a = b = [1e200, 1e200] → nan: every square overflows to infinity, and infinity over infinity is NaN. Dividing each vector by its largest absolute entry first gives the same cosine without overflow.
• a = b = [1e-200, 1e-200] → ZeroDivisionError: the squares underflow to 0.0, so a nonzero vector looks like a zero vector. The same rescaling fixes it.

Common mistakes

• Returning the raw dot product without dividing by the magnitudes — then length dominates and it is no longer cosine.
• Dividing by zero on a zero vector — guard it (define the similarity as 0 or handle explicitly).
• Assuming the inputs are the same length without checking.

Follow-ups an interviewer asks next

You compare one query against a million stored embeddings.

Normalize every stored vector to unit length once, at write time, and the query once per search. Cosine then reduces to a plain dot product, so no comparison pays for norms. A full scan is still O(N·d), so at scale an approximate nearest-neighbor index such as HNSW or IVF answers top-k queries without touching every vector, trading a little recall for speed.

Make it numerically robust.

Compare with a tolerance, and clamp the result to [-1, 1] before any acos, since rounding can land just outside. To avoid overflow and underflow, scale each vector by its largest absolute entry before squaring; cosine is unchanged by scaling. In Python, math.fsum sums many terms more accurately, and math.hypot(*v) computes a norm without intermediate overflow.

When should you use the dot product or Euclidean distance instead?

On unit-length vectors all three give the same ranking: the dot product equals cosine, and squared Euclidean distance equals 2 − 2·cos. They diverge when length carries meaning, for example in a model trained so that magnitude encodes something. Use the metric the embedding model was trained with; its documentation usually names it.

The vectors are sparse, like TF-IDF over a large vocabulary.

Store each one as a map from index to nonzero value, iterate over the smaller map, and look each key up in the other. The dot product then costs O(min(nnz(a), nnz(b))) instead of O(d), and each document’s norm can be computed once and stored. Most entries are zero, so skipping them is where nearly all of the saving comes from.

Score one query against a whole matrix at once.

Stack the stored vectors as the rows of a matrix and divide each row by its norm once. With NumPy, scores = M @ (q / np.linalg.norm(q)) then gives every cosine in one matrix-vector product, which runs in optimized native code instead of a Python loop. To pick the top k without sorting all N scores, np.argpartition(-scores, k)[:k] finds them in O(N) and only those k need sorting. A zero row still divides by zero, so drop or flag zero vectors when you normalize.

Where this shows up

Cosine similarity is the standard score for comparing embeddings — semantic search, RAG retrieval, clustering, deduplication, recommendations. On vectors that are already unit-normalized (many embedding models return these) it reduces to a plain dot product, which is exactly why vector databases can rank by dot product and get cosine ordering for free.

Before embeddings. The classic vector space model of information retrieval represented each document and query as a TF-IDF vector over the vocabulary and ranked documents by cosine. Dividing out length was the point there too: without it, a long document would beat a short, focused one simply by repeating more words. Item-based recommenders use the same score between columns of a user-item rating matrix, often in an “adjusted” form that first subtracts each user’s average rating, so that a generous rater and a harsh one who rank items the same way still look alike.

Scores are not probabilities. A cosine of 0.8 is not “80% similar”, and the same number means different things for different embedding models, because each model spreads its vectors over the sphere differently. A retrieval threshold, such as “only pass chunks above 0.75 to the model”, has to be chosen per model from labeled pairs of relevant and irrelevant results, and chosen again when the model changes.

Similarity versus distance. Libraries often report cosine distance, 1 − cos, so a smaller number is better; reading it as a similarity flips every ranking. It is also not a true metric, because it can violate the triangle inequality. When an algorithm needs a real metric, use the angle itself, acos(cos), or, on unit vectors, Euclidean distance, which equals sqrt(2 − 2·cos) and so ranks identically.

Check yourself

Reference solution

import math

def cosine_similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))      # alignment
    norm_a = math.sqrt(sum(x * x for x in a))   # length of a
    norm_b = math.sqrt(sum(y * y for y in b))   # length of b
    return dot / (norm_a * norm_b)


print(cosine_similarity([1, 2, 3], [1, 2, 3]))  # -> 1.0
print(cosine_similarity([1, 0], [0, 1]))        # -> 0.0

function cosineSimilarity(a: number[], b: number[]): number {
  let dot = 0, na = 0, nb = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];   // alignment
    na += a[i] * a[i];    // |a|^2
    nb += b[i] * b[i];    // |b|^2
  }
  return dot / (Math.sqrt(na) * Math.sqrt(nb));
}

console.log(cosineSimilarity([1, 2, 3], [1, 2, 3]));  // -> 1
console.log(cosineSimilarity([1, 0], [0, 1]));        // -> 0