Pick the best few from millions

Use a funnel. A cheap, fast stage narrows millions to a few hundred plausible candidates; a heavy, accurate stage ranks those few hundred precisely. Cheap-and-wide, then expensive-and-narrow — the pattern behind every large recommender.

A request and a funnel

A Recs Gateway runs a two-stage funnel for every request: candidate generation (millions → hundreds, fast) then ranking (hundreds → ordered, accurate), with filtering before the final list ships to the user. The shape is the whole idea.

Embeddings for users and items

The system learns embeddings: users and items map to vectors in a shared space where a user sits near the items they’d engage with. These come from training on past engagement, so proximity encodes learned taste — not hand-typed tags.

Two-tower retrieval with ANN

A two-tower model has a user tower (computed live from the request) and an item tower (item vectors precomputed offline and stored in the Item Index). Candidate Generation embeds the user, then runs ANN search to pull the nearest few hundred items in milliseconds. Recall stage: don’t miss the good ones.

The heavy ranking model

A heavy Ranking Model scores each candidate’s probability of engagement (click, watch, purchase) using rich features — user history, item attributes, context (time, device), cross features. It’s far too expensive to run over millions, but perfect over a few hundred. Precision stage: order them right.

A Feature Store serves ranking its features and has two halves that must agree: an offline pipeline computes features in batch (for training), and an online store serves the freshest values at request time (for serving) — from a single definition, so both see the same numbers.

Filters, rules and diversity

A Filters & Rules stage takes the ranked list and makes it shippable: drop already-seen and blocked/policy-violating items, enforce diversity (don’t show ten near-identical items), and apply business rules (freshness, sponsored slots, fairness). Then the final feed goes to the user.

Log engagement, retrain, repeat

Every impression and interaction is logged to the Engagement Log. A Training Pipeline periodically retrains the retrieval and ranking models on that data and refreshes the feature store — so the system learns from what worked and adapts as taste shifts. The loop is the product.

Design a Recommendation System — A Guided System Design

System Design · step by stepDesign a Recommendation System

Step 1 / 9

RUN IT YOURSELF

Item-based collaborative filtering

"Because you liked X" works by finding items similar to ones you already liked — item-item cosine similarity over a rating/feature matrix. Here it is in real Python, running live. Read the comments, edit the catalog, and hit Run.

HOW TO READ THE CODE — 4 IDEAS

Each item is a feature vector (here: how action / comedy / romance it is).
Two items are similar if their vectors point the same way — cosine similarity.
Score each unseen item by its best similarity to something you liked (steps 1–2).
Recommend the top-k, never re-suggesting what you have seen (step 3).

CPython · WebAssembly

# A recommender suggests items you haven't seen, ranked by similarity
# to items you already liked: item-item COSINE SIMILARITY over a tiny
# catalog of feature vectors.
import math

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    na = math.sqrt(sum(x * x for x in a)); nb = math.sqrt(sum(y * y for y in b))
    return dot / (na * nb) if na and nb else 0.0

def recommend(liked, catalog, k):
    scores = {}
    for item, vec in catalog.items():
        if item in liked:
            continue                                          # skip what you've seen
        # STEP 1-2 - score by best similarity to a liked item.
        scores[item] = max(cosine(vec, catalog[l]) for l in liked)
    # STEP 3 - return the top-k highest-scoring items.
    return sorted(scores, key=scores.get, reverse=True)[:k]

# items as feature vectors: [action, comedy, romance]
catalog = {
    "Die Hard":     [0.9, 0.2, 0.0],
    "Rush Hour":    [0.8, 0.7, 0.0],
    "Notting Hill": [0.0, 0.5, 0.9],
    "John Wick":    [0.95, 0.1, 0.0],
}
print("because you liked Die Hard:", recommend(["Die Hard"], catalog, 2))

Finished this one? 0 / 5 AI System Designs done