Vibe Engines Learn
CODING CHALLENGE · N°10

Cross-Entropy Loss

Easy AI EngineeringDeep LearningLoss Functions

The loss that trains almost every classifier and language model. It measures how surprised the model was by the right answer: if it put high probability on the true class, the loss is near 0; if it was confidently wrong, the loss explodes. No numpy — just a logarithm.

The problem

Implement cross-entropy loss for a single example. You are given a predicted probability distribution p (a list that sums to ~1) and a target. The target is either an integer class index or a one-hot list. Return the negative log-probability the model assigned to the true class: -log(p[target]) for an index, or -Σ tᵢ·log(pᵢ) for a one-hot target. Guard the logarithm with a tiny eps so a probability of 0 gives a large finite loss instead of -inf.

EXAMPLE 1
Input p=[0.0, 1.0, 0.0], target=1
Output 0.0
all probability on the right class → no surprise, no loss
EXAMPLE 2
Input p=[0.25, 0.25, 0.25, 0.25], target=0
Output ≈1.3863
a uniform guess over K=4 → loss = log(4)
EXAMPLE 3
Input p=[0.999, 0.0005, 0.0005], target=1
Output ≈7.6
confident and wrong → large loss
CONSTRAINTS
  • For an integer target, the loss is -log(p[target]).
  • For a one-hot list target, the loss is -Σ tᵢ·log(pᵢ).
  • Clamp each probability to at least eps = 1e-12 before taking its log, so log(0) never happens.
  • Pyodide has no numpy — use the math stdlib and plain lists only.
SOLVE IT YOURSELF

Your turn — write it

Edit the stub, hit Run (or ⌘/Ctrl + Enter), and watch the hidden tests. Stuck? the hints are right above and Reveal solution is one click away.

YOUR TASK

Implement cross_entropy(p, target, eps=1e-12). Return the negative log-probability of the true class — accepting either an integer index or a one-hot list as the target.

HINTS — 4 IDEAS
  1. The whole loss for an integer target is one line: -log(p[target]).
  2. Protect the log with max(p[target], eps) so a zero probability gives a big finite number, not -inf.
  3. If the target is a list (one-hot), sum -tᵢ·log(pᵢ) over all classes — only the tᵢ=1 term survives.
  4. Perfect confidence (p on the true class = 1) gives log(1) = 0; a uniform guess over K classes gives log(K).
CPython · WebAssembly

Explore the topic

See this challenge alongside everything else on the same subject — handbooks, system designs, algorithms and tools, in one place.

LLM Engineering