Classification vs Regression: Which Bucket, or How Much?
Classification sorts input into a fixed set of discrete buckets — cat or dog, spam or not — and is scored by accuracy. Regression predicts a continuous number on a line, like temperature or price, and is scored by how far off the prediction lands.
Classification sorts an input into one of a fixed set of discrete categories — spam or not spam — drawing boundaries and scoring with something like cross-entropy loss. Regression predicts a continuous quantity — a price, a temperature — fitting a curve and scoring with squared error. The twist is that a classifier usually outputs a number first, a probability, and only thresholds it into a bucket afterward — which is exactly why logistic regression, despite the name, is a classifier: a fixed set of kinds means classify, a quantity on a continuum means regress.
Transcript
You show a model a photo of a dog. Ask 'WHAT animal is this?' — you want a bucket, a label. Ask 'HOW OLD is it?' — you want a number. Same photo, two totally different questions — and that split is the whole difference between classification and regression. It changes everything downstream.
Classification sorts each input into one of a fixed set of BUCKETS. Cat, dog, or bird. Spam or not. The output is a discrete label, a category. Under the hood it draws boundaries between the buckets, and you score it by how often it drops things in the right one — accuracy, precision, recall.
Regression predicts a continuous NUMBER instead — a point anywhere on a line. Tomorrow's temperature, a house price, a person's age. There are no buckets; it fits a curve through the data, and you score it by how FAR off the number is — the average error, not right-or-wrong.
Here's the twist: the line between them is blurry. A classifier usually outputs a NUMBER first — a probability — then thresholds it into a bucket. That's why 'logistic regression' is actually a classifier. And you can turn any regression into classification by bucketing the number: cheap, mid, or pricey.
So the framing you pick isn't cosmetic — it sets the output layer, the loss function, and the metric. Categories use cross-entropy; numbers use squared error. Ask what the answer really is. A fixed set of distinct kinds? Classify. A quantity on a continuum, where magnitude and order matter? Regress.
Same data, two questions: drop it in a bucket, or place it on a number line. One predicts a KIND, the other a QUANTITY. So next time you frame an ML problem, ask this first: which one — or how much? That single choice decides your output, your loss, and your metric.