Why can't you maximize precision and recall at the same time?

Most classifiers output a suspicion score, not a yes or no, and a threshold decides how much suspicion is enough to flag something. Lowering the threshold flags more items, so you catch more real positives (recall rises) but also mislabel more negatives (precision falls). Raising it does the reverse. Moving the threshold does not make the model better - it only chooses which mistake you make more of.

What is F-beta and how is it different from F1?

F-beta is the weighted version: (1 + beta squared) * P * R / (beta squared * P + R). Beta sets how many times more you care about recall than precision. Beta = 1 gives F1, treating them equally. Beta = 2 (F2) weights recall higher, which suits screening problems where a miss is costly. Beta = 0.5 (F0.5) weights precision higher, which suits filters where a false alarm is costly.

Why is accuracy a misleading metric for imbalanced data?

Accuracy is (TP + TN) / total, so when negatives dominate, the TN term swamps everything else. If only 15% of emails are spam, a filter that flags nothing at all scores 85% accuracy while catching zero spam. Precision, recall and the precision-recall curve ignore true negatives, which is exactly why they stay informative under class imbalance.

Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path

LABS / PRECISION vs RECALL

Which mistake can you live with?

Q: What is the difference between precision and recall?

Precision asks: of everything the model flagged as positive, how much was actually positive? It is TP / (TP + FP), a false-alarm score. Recall asks: of everything that was actually positive, how much did the model catch? It is TP / (TP + FN), a miss score. Precision is silent about what you missed; recall is silent about how many false alarms you raised.

Q: What is the F1 score and when should I use it?

F1 is the harmonic mean of precision and recall: 2PR / (P + R). Because it is harmonic rather than arithmetic, it punishes lopsided models - a classifier with precision 1.0 and recall 0.1 gets an F1 of about 0.18, not 0.55. Use F1 as a single comparison number when both mistakes genuinely matter and you have no strong reason to favour one over the other.

Q: How do you choose the right decision threshold?

By the cost of each mistake, not by chasing accuracy. Ask which error you can live with. A cancer screen wants high recall, because a missed case is catastrophic and a false alarm only costs a follow-up test. A delete-forever spam filter wants high precision, because destroying one real email is unacceptable. Pick the threshold that matches that answer, then report the precision and recall it actually delivers.

A spam filter doesn't answer yes or no — it answers how suspicious. You pick the line. Drag it left and you catch every spam but bury real mail; drag it right and every flag is correct but the spam pours in. There is no setting that wins both. Choose your mistake.

THE GIST · 20 SECONDS

Precision = of everything you flagged, how much was really spam (TP / (TP + FP)) — a false-alarm score. Recall = of all the real spam, how much did you catch (TP / (TP + FN)) — a miss score. One threshold controls both, in opposite directions. F1 is their harmonic mean. Accuracy lies when one class is rare.

Precisiontrust a flag
Recallmiss nothing
Thresholdthe one dial
F1harmonic mean

1Drag left. Flag aggressively. Recall climbs toward 100% — and precision falls off a cliff.

2Drag right. Flag only the obvious. Precision reaches 100% — and most spam walks straight past.

3Find the balance. Watch F1 peak somewhere in the middle, and the dot ride the PR curve.

4Hit "flag nothing". 85% accuracy. Zero spam caught. That's the trap.

Drag the threshold line (or use the slider) — everything to its right gets flagged as spam.

THE PR CURVE

Every threshold is one point on this curve. You cannot move off it — you can only slide along it. Up-and-right is a better model; along the line is just a different choice.

The dashed floor at 0.15 is what random guessing scores here, because 15% of the mail really is spam.

📬

THE FILTER 5,000 emails · 750 spam

Slide the threshold. Everything scoring above it lands in the spam folder; everything below it lands in the inbox. Two of those four outcomes are mistakes — and they cost different things.

THRESHOLD 0.50

TP · caught spam—

FP · real mail lost—

FN · spam missed—

TN · mail delivered—

PRECISION—

RECALL—

F1—

ACCURACY—

A false positive deletes an email someone actually needed.

A false negative drops a phishing link into the inbox.

UNDER THE HOOD

What you just played, written down

A classifier that's "90% accurate" can still be failing you badly, because accuracy hides the detail that matters most: there are two completely different ways to be wrong.

Two ways to be wrong

A spam filter can flag a real email you needed — a false alarm — or it can let actual spam slide into your inbox — a miss. Lumping those together into one accuracy number throws away the information you need to judge the model. Precision and recall separate them.

Precision is a false-alarm score. Of everything flagged, how much was really spam? High precision means the model rarely cries wolf — but it says nothing about the mail it never flagged. Flag only the five most obvious spams out of a thousand and precision looks perfect while you miss almost everything.
Recall is a miss score. Of all the real spam, how much did you catch? High recall means little slips through — but it says nothing about the good mail caught along the way. Flag every email and recall is a perfect 1.0, and the filter is useless.
They are two ends of one dial. The threshold decides how much suspicion is enough. Turn it down and recall climbs while precision falls; turn it up and precision climbs while recall falls. Moving it never improves the classifier — it only chooses which mistake you'd rather make more of.
Accuracy hides the whole thing. With 750 spam in 5,000 emails, flagging nothing scores 85% accuracy and catches nothing. Precision and recall ignore true negatives, which is exactly why they survive class imbalance.

THE QUESTION THAT ACTUALLY DECIDES IT

Not "how accurate is this model." Ask: for this problem, is it worse to miss a real positive, or to raise a false alarm? Set the threshold to match that answer, and own the tradeoff you chose.

The formulas in code

TP = flagged and really spam
FP = flagged but really good mail   # false alarm
FN = not flagged but really spam    # miss
TN = not flagged and really good mail

precision = TP / (TP + FP)
recall    = TP / (TP + FN)
accuracy  = (TP + TN) / (TP + FP + FN + TN)

f1    = 2 * precision * recall / (precision + recall)
# weighted: beta > 1 favours recall, beta < 1 favours precision
fbeta = (1 + b**2) * precision * recall / (b**2 * precision + recall)

FLAG NOTHING85.0%accuracy, 0% recall

BEST F1 HERE0.836at threshold 0.60

WHY HARMONIC, NOT AVERAGE

At threshold 0.84 this filter hits precision 1.000 and recall 0.261. A plain average would call that 0.63 — respectable. F1 calls it 0.414, because the harmonic mean is dragged down by the weaker number. That's the point: F1 refuses to reward a lopsided model.

F1 vs F_β

F1 assumes the two mistakes cost the same. They almost never do. F_β lets you say how much more you care about recall: β = 2 (F2) weights recall roughly twice as heavily — the screening setting; β = 0.5 (F0.5) weights precision — the delete-forever-filter setting. If you can write down the dollar or harm cost of an FP and an FN, skip F-scores entirely and optimise expected cost directly.

⚠ Optimise recall when a miss is catastrophic

A cancer screening test wants recall. Missing a real case can be fatal, so you accept more false alarms — healthy patients sent for a follow-up — in exchange for almost never missing someone who is sick. Same logic for fraud detection and safety filters.

↔ Optimise precision when a false alarm is unacceptable

A delete-forever spam filter wants precision. Destroying one real email is unrecoverable, so you'd rather let some spam through. Same logic for auto-banning accounts, or an agent that takes an irreversible action on its own.

★ Report the pair, not the peak

"Precision 0.86 at recall 0.81" is a claim you can check. "94% accurate" usually isn't. When comparing models, compare whole PR curves — a model that dominates at every threshold is genuinely better; a model that merely sits elsewhere on the same curve is not.

GO DEEPER

ML Fundamentals — where confusion matrices and metrics fit in the wider training loop.
51 LLM Evals Interview Questions — precision/recall questions, asked the way interviewers ask them.
Agent Evals — scoring systems whose "positives" are tool calls and task completions.
Evals in CI — pinning a threshold so a metric regression fails the build.
Gradient Descent and Perceptron Playground — how the scores you just thresholded get learned in the first place.
More at all labs and the handbook library. External: scikit-learn metrics docs, Wikipedia.

THE PICTURE

The picture: one line cuts two piles

The lab’s data in ten bins. Spam stacks up on the right and real mail on the left, but the two piles overlap between about 0.3 and 0.8, and no line can cut an overlap cleanly. Everything right of the line is flagged, so the spam there is caught (TP) and the mail there is lost (FP). Each pile is drawn to its own scale: the mail pile is 4,250 emails against 750 spam, which is why 100 lost emails is a thin sliver here and still a real cost.

BY THE NUMBERS

Worked example: seven thresholds on the same 5,000 emails

Take the 0.60 row. Of the 750 spam, 610 score 0.60 or higher: TP = 610, FN = 140. Of the 4,250 real emails, 100 score that high: FP = 100. Precision is 610 / (610 + 100) = 0.859. Recall is 610 / 750 = 0.813. F1 is 2 × 0.859 × 0.813 / (0.859 + 0.813) = 0.836. Accuracy is (610 + 4,150) / 5,000 = 95.2%.

Threshold	TP	FP	FN	Precision	Recall	F1	Accuracy
0.30	747	1,595	3	0.319	0.996	0.483	68.0%
0.50	694	323	56	0.682	0.925	0.786	92.4%
0.60	610	100	140	0.859	0.813	0.836	95.2%
0.62	587	76	163	0.885	0.783	0.831	95.2%
0.70	469	21	281	0.957	0.625	0.756	94.0%
0.84	196	0	554	1.000	0.261	0.414	88.9%
1.00	0	0	750	undefined	0.000	0.000	85.0%

F1 peaks at 0.60. Accuracy peaks one notch later, at 0.62 (95.22% against 95.20%), because accuracy treats both mistakes as equal. Now give them prices. Say losing a real email costs 10 times a missed spam. Minimising 10·FP + FN moves the line to 0.74: 397 spam caught, 9 emails lost. Flip it, so a missed phishing link costs 10 times a lost email, and the line drops to 0.46: 714 caught, 476 lost. F2 lands between, at 0.50 to 0.52; F0.5 lands at 0.66. Same model every time. Only the price list changed.

TRADE-OFFS

Cost and trade-offs: computing the numbers

For n scored examples. Choosing a threshold is cheap; seeing the whole curve costs a sort or a histogram.

Case	Time	Space	Why
One threshold	O(n)	O(1)	One pass to fill the four cells.
Exact PR curve	O(n log n)	O(n)	Sort by score, then sweep once; each new example moves one count.
Binned curve, like this lab	O(n + B)	O(B)	With B = 50 bins the curve has 51 points, 0.02 apart. Histograms from different shards just add up.
Average precision (area under PR)	O(n log n)	O(n)	One number for the whole curve. Good for ranking models, but it hides where on the curve you will operate.
Precision@k in search	O(n log k)	O(k)	Keep the top k in a heap. Here k plays the role of the threshold.

The trade-off is resolution against memory. Exact curves need every score kept and sorted. Binned curves fit any data size in fixed memory, but they can only place the threshold on a bin edge.

IN THE WILD

Where this shows up: production and interviews

When the base rate moves
Recall is a property of the spam alone; precision also depends on how much spam there is. At 0.60 this filter catches 81.3% of spam and flags 2.35% of real mail. At 15% spam that gives precision 0.859. If spam fell to 5% of traffic, the same filter would give 0.645; at 1.5%, 0.345. Recall would not move. A threshold tuned on a balanced test set can disappoint in production for exactly this reason.
Search and retrieval for LLMs
A retriever is judged by recall@k: did the right passage make the top k? A reranker or the model then cares about precision among those k. Raising k is the threshold dial again: more recall, more noise in the prompt.
Detectors and leaderboards
Object detection reports average precision, the area under the PR curve, averaged across classes as mAP. Moderation and fraud teams instead fix an operating point, such as a minimum precision, and report the recall they get there.
Interview angles
Expect: “Why not accuracy?” (flag nothing: 85%). “ROC or PR when positives are rare?” PR, because the false-positive rate divides by all 4,250 real emails: at 0.50 it is a calm-looking 0.076 while precision is only 0.682. “How do you pick the threshold?” By the cost of each mistake, then report both numbers.

QUICK CHECK

Did it stick?

See every answer, with the reasoning

A spam filter flags 100 emails and 95 of them really are spam, but 400 spam emails got through. What are precision and recall?
Answer: Precision 95%, recall 19% — Precision = TP / (TP + FP) = 95 / 100 = 95% — of what you flagged, almost all was right. Recall = TP / (TP + FN) = 95 / (95 + 400) = 19% — of all the real spam, you only caught a fifth. High precision, terrible recall.
Only 2% of transactions are fraud. A model predicts "not fraud" for every single transaction. What does it score?
Answer: 98% accuracy, 0% recall — accuracy is meaningless here — It gets all 98% of the legitimate transactions right and every fraud wrong, so accuracy is 98% while recall is 0%. When one class dominates, the true-negative term swamps accuracy — which is exactly why you report precision and recall instead.
A cancer screening test: which metric do you push, and what do you pay for it?
Answer: Recall — accept more false alarms to avoid missing a real case — A missed case can be fatal; a false alarm costs a follow-up test. So you lower the threshold, push recall up, and knowingly accept more false positives. The right metric is set by the cost of each mistake, never by which number looks best.
Precision is 1.00 and recall is 0.20. What is the F1 score?
Answer: about 0.33 — the harmonic mean — F1 = 2PR / (P + R) = 2 × 1.00 × 0.20 / 1.20 ≈ 0.333. The harmonic mean is dragged toward the smaller number, so a lopsided model cannot hide behind its strong metric — that is the whole reason F1 is used instead of a plain average.

FAQ

Precision and recall, answered

What is the difference between precision and recall?

Precision = TP / (TP + FP): of everything you flagged, how much was actually positive — a false-alarm score. Recall = TP / (TP + FN): of everything that actually was positive, how much did you catch — a miss score. Precision is silent about what you missed; recall is silent about the false alarms you raised.

Why can't you maximize both at once?

They're two ends of one dial: the threshold. Lower it and you flag more, so recall rises and precision falls. Raise it and the reverse happens. Moving the threshold doesn't make the model better — it just picks which mistake you make more of.

What is the F1 score and when should I use it?

The harmonic mean, 2PR / (P + R). Because it's harmonic, it punishes lopsided models: precision 1.0 with recall 0.1 scores about 0.18, not 0.55. Use it as a single comparison number when both mistakes genuinely matter and you have no reason to favour one.

What is F_β and how is it different?

The weighted version: (1 + β²)·P·R / (β²·P + R). β says how many times more you care about recall. β = 1 is F1; F2 favours recall (screening); F0.5 favours precision (delete-forever filters).

Why is accuracy misleading on imbalanced data?

Accuracy is (TP + TN) / total, so when negatives dominate, the true-negative term swamps everything. If 15% of mail is spam, a filter that flags nothing scores 85% accuracy and catches zero spam. Precision and recall ignore TN, which is why they survive imbalance.

How do I choose the right threshold?

By the cost of each mistake, not by chasing accuracy. A cancer screen wants recall — a miss is catastrophic, a false alarm costs a follow-up test. A delete-forever spam filter wants precision — destroying one real email is unacceptable. Pick to match, then report the precision and recall you actually deliver.

Finished this one? 0 / 61 Labs done

NextBias vs Variance: Fit the Curve, Watch It Overfit Next on the “Judging a model” ladder · rung 2 of 3

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

ML Foundations