Discriminative vs Generative Models: A Critic, or an Artist?
Two opposite ways a model learns from data: a discriminative model only learns the boundary between classes, while a generative model learns the full distribution of each class well enough to create new samples — which is exactly what makes modern generative AI generative.
Discriminative models (logistic regression, SVMs, most classifiers) only learn P(y|x) — the boundary between classes — pouring all their capacity into telling things apart, which usually makes them sharper classifiers. Generative models (Naive Bayes, GANs, diffusion, LLMs) learn P(x|y) — the full distribution each class comes from — which is a harder job and often a weaker classifier, but it means they can also generate new samples, flag outliers, and learn from less labeled data.
Transcript
Here are two ways to teach a model the difference between a cat and a dog. One learns only the LINE between them. The other learns to PAINT each one. Same data, opposite philosophies — and which you pick shapes everything the model can do. Discriminative, or generative?
A discriminative model learns just the boundary. Given an image, it answers one question — cat or dog? — and models that decision line directly. It pours all its capacity into where the classes differ, and ignores everything else. It's a critic who only tells them apart.
A generative model does something harder. It learns the full distribution of each class — for an image model, what a cat and a dog actually look like. It's an artist who models each class — the strongest ones can paint a brand-new one. To classify, it asks: which does this look more like?
So the discriminative model is the sharper classifier — it only models the boundary, so it usually wins on pure accuracy. The generative model has a harder job, so it's often a weaker classifier — but it can GENERATE new data, flag outliers, and learn from less labeled data.
And that's the whole reason generative AI exists. GANs and diffusion learn the distribution of images; an LLM learns the distribution of language. Modeling that distribution is exactly what lets them CREATE — an image or a sentence that never existed. A discriminative model can only judge; a generative one can make.
So ask what you need. Only to tell classes apart? Go discriminative — sharper and cheaper. To generate, or spot the unseen? Go generative. It comes down to one question: do you need to judge, or to paint? A critic, or an artist?