Vibe Engines Learn
CODING CHALLENGE · N°12

TF-IDF

Medium AI EngineeringNLPRetrieval

The scoring that ran search for decades — and still seeds hybrid retrieval today. It rewards a word that is frequent in one document but rare across the corpus, and shrugs off words that appear everywhere. No numpy, just a logarithm.

The problem

Implement TF-IDF: the weight of a term in one document of a corpus. The corpus is a list of documents, each a list of tokens; doc_index selects the document. Term frequency is the term's share of that document: tf = count(term in doc) / len(doc). Inverse document frequency uses how many documents contain the term (df) out of N total: idf = log(N / df). Return tf · idf. If the term is absent from the document, its weight is 0.

EXAMPLE 1
Input corpus=[['a','b'],['a','c'],['a','d']], doc_index=0, term='a'
Output 0.0
'a' is in every doc → idf = log(1) = 0 → weight 0
EXAMPLE 2
Input corpus=[['a','a','b','c'],['d'],['e']], doc_index=0, term='a'
Output ≈0.5493
tf = 2/4, idf = log(3) → a rare, frequent term scores high
EXAMPLE 3
Input corpus=[['a','b'],['c']], doc_index=0, term='z'
Output 0.0
term not in the document → weight 0
CONSTRAINTS
  • Term frequency: tf = count(term in doc) / len(doc).
  • Document frequency df = number of documents containing the term; idf = log(N / df) with N = len(corpus) (natural log).
  • If the term is absent from the document, return 0 (so you never divide by df = 0).
  • Pyodide has no numpy — use the math stdlib and plain lists only.
SOLVE IT YOURSELF

Your turn — write it

Edit the stub, hit Run (or ⌘/Ctrl + Enter), and watch the hidden tests. Stuck? the hints are right above and Reveal solution is one click away.

YOUR TASK

Implement tf_idf(corpus, doc_index, term). Return tf · idf for the term in the selected document — the term's share of that doc, scaled by how rare it is across the corpus.

HINTS — 4 IDEAS
  1. tf = (times the term appears in the doc) / (number of tokens in the doc).
  2. If tf is 0 the whole weight is 0 — return early, and you avoid a df = 0 divide.
  3. df = how many documents contain the term at all; idf = log(N / df) with N = len(corpus).
  4. A term in every document has df = N, so idf = log(1) = 0 — it carries no weight.
CPython · WebAssembly

Explore the topic

See this challenge alongside everything else on the same subject — handbooks, system designs, algorithms and tools, in one place.

RAG & Retrieval LLM Engineering