Vibe Engines Learn
LABS  /  PRECISION vs RECALL

Which mistake can you live with?

A spam filter doesn't answer yes or no — it answers how suspicious. You pick the line. Drag it left and you catch every spam but bury real mail; drag it right and every flag is correct but the spam pours in. There is no setting that wins both. Choose your mistake.

THE GIST · 20 SECONDS

Precision = of everything you flagged, how much was really spam (TP / (TP + FP)) — a false-alarm score. Recall = of all the real spam, how much did you catch (TP / (TP + FN)) — a miss score. One threshold controls both, in opposite directions. F1 is their harmonic mean. Accuracy lies when one class is rare.

  • Precisiontrust a flag
  • Recallmiss nothing
  • Thresholdthe one dial
  • F1harmonic mean
1Drag left. Flag aggressively. Recall climbs toward 100% — and precision falls off a cliff.
2Drag right. Flag only the obvious. Precision reaches 100% — and most spam walks straight past.
3Find the balance. Watch F1 peak somewhere in the middle, and the dot ride the PR curve.
4Hit "flag nothing". 85% accuracy. Zero spam caught. That's the trap.

Drag the threshold line (or use the slider) — everything to its right gets flagged as spam.

THE PR CURVE

Every threshold is one point on this curve. You cannot move off it — you can only slide along it. Up-and-right is a better model; along the line is just a different choice.

The dashed floor at 0.15 is what random guessing scores here, because 15% of the mail really is spam.

UNDER THE HOOD

What you just played, written down

A classifier that's "90% accurate" can still be failing you badly, because accuracy hides the detail that matters most: there are two completely different ways to be wrong.

Two ways to be wrong

A spam filter can flag a real email you needed — a false alarm — or it can let actual spam slide into your inbox — a miss. Lumping those together into one accuracy number throws away the information you need to judge the model. Precision and recall separate them.

  1. Precision is a false-alarm score. Of everything flagged, how much was really spam? High precision means the model rarely cries wolf — but it says nothing about the mail it never flagged. Flag only the five most obvious spams out of a thousand and precision looks perfect while you miss almost everything.
  2. Recall is a miss score. Of all the real spam, how much did you catch? High recall means little slips through — but it says nothing about the good mail caught along the way. Flag every email and recall is a perfect 1.0, and the filter is useless.
  3. They are two ends of one dial. The threshold decides how much suspicion is enough. Turn it down and recall climbs while precision falls; turn it up and precision climbs while recall falls. Moving it never improves the classifier — it only chooses which mistake you'd rather make more of.
  4. Accuracy hides the whole thing. With 750 spam in 5,000 emails, flagging nothing scores 85% accuracy and catches nothing. Precision and recall ignore true negatives, which is exactly why they survive class imbalance.
THE QUESTION THAT ACTUALLY DECIDES IT

Not "how accurate is this model." Ask: for this problem, is it worse to miss a real positive, or to raise a false alarm? Set the threshold to match that answer, and own the tradeoff you chose.

The formulas in code

TP = flagged and really spam
FP = flagged but really good mail   # false alarm
FN = not flagged but really spam    # miss
TN = not flagged and really good mail

precision = TP / (TP + FP)
recall    = TP / (TP + FN)
accuracy  = (TP + TN) / (TP + FP + FN + TN)

f1    = 2 * precision * recall / (precision + recall)
# weighted: beta > 1 favours recall, beta < 1 favours precision
fbeta = (1 + b**2) * precision * recall / (b**2 * precision + recall)
FLAG NOTHING85.0%accuracy, 0% recall
BEST F1 HERE0.836at threshold 0.60
WHY HARMONIC, NOT AVERAGE

At threshold 0.84 this filter hits precision 1.000 and recall 0.261. A plain average would call that 0.63 — respectable. F1 calls it 0.414, because the harmonic mean is dragged down by the weaker number. That's the point: F1 refuses to reward a lopsided model.

F1 vs Fβ

F1 assumes the two mistakes cost the same. They almost never do. Fβ lets you say how much more you care about recall: β = 2 (F2) weights recall roughly twice as heavily — the screening setting; β = 0.5 (F0.5) weights precision — the delete-forever-filter setting. If you can write down the dollar or harm cost of an FP and an FN, skip F-scores entirely and optimise expected cost directly.

⚠ Optimise recall when a miss is catastrophic

A cancer screening test wants recall. Missing a real case can be fatal, so you accept more false alarms — healthy patients sent for a follow-up — in exchange for almost never missing someone who is sick. Same logic for fraud detection and safety filters.

↔ Optimise precision when a false alarm is unacceptable

A delete-forever spam filter wants precision. Destroying one real email is unrecoverable, so you'd rather let some spam through. Same logic for auto-banning accounts, or an agent that takes an irreversible action on its own.

★ Report the pair, not the peak

"Precision 0.86 at recall 0.81" is a claim you can check. "94% accurate" usually isn't. When comparing models, compare whole PR curves — a model that dominates at every threshold is genuinely better; a model that merely sits elsewhere on the same curve is not.

GO DEEPER
QUICK CHECK

Did it stick?

FAQ

Precision and recall, answered

What is the difference between precision and recall?

Precision = TP / (TP + FP): of everything you flagged, how much was actually positive — a false-alarm score. Recall = TP / (TP + FN): of everything that actually was positive, how much did you catch — a miss score. Precision is silent about what you missed; recall is silent about the false alarms you raised.

Why can't you maximize both at once?

They're two ends of one dial: the threshold. Lower it and you flag more, so recall rises and precision falls. Raise it and the reverse happens. Moving the threshold doesn't make the model better — it just picks which mistake you make more of.

What is the F1 score and when should I use it?

The harmonic mean, 2PR / (P + R). Because it's harmonic, it punishes lopsided models: precision 1.0 with recall 0.1 scores about 0.18, not 0.55. Use it as a single comparison number when both mistakes genuinely matter and you have no reason to favour one.

What is Fβ and how is it different?

The weighted version: (1 + β²)·P·R / (β²·P + R). β says how many times more you care about recall. β = 1 is F1; F2 favours recall (screening); F0.5 favours precision (delete-forever filters).

Why is accuracy misleading on imbalanced data?

Accuracy is (TP + TN) / total, so when negatives dominate, the true-negative term swamps everything. If 15% of mail is spam, a filter that flags nothing scores 85% accuracy and catches zero spam. Precision and recall ignore TN, which is why they survive imbalance.

How do I choose the right threshold?

By the cost of each mistake, not by chasing accuracy. A cancer screen wants recall — a miss is catastrophic, a false alarm costs a follow-up test. A delete-forever spam filter wants precision — destroying one real email is unacceptable. Pick to match, then report the precision and recall you actually deliver.

YOUR CHOICE

—

Next → a threshold is a decision you own. Write it into your eval suite, pin it in CI, and report the precision and recall it delivers — not the accuracy that hides them.

Finished this one? 0 / 61 Labs done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

ML Foundations

More Labs

Explore more from Vibe Engines

Handbooks →
Roadmaps →
System Designs →
AI System Designs →
Algorithms →
Paper Breakdowns →