Classifier-free
guidance.

A diffusion model generates by starting from pure noise and, step by step, predicting and removing a little noise until an image emerges. To make it follow a prompt, you condition that noise prediction on the text. But conditioning alone is often too gentle — the model drifts, hedges, produces something loosely on-topic. You want a dial: how hard should it obey the prompt?

The first answer, classifier guidance, borrowed a separate image classifier and pushed each denoising step toward whatever the classifier thought matched the label — effective, but it meant training and running an extra network on noisy images, an ungainly failure-prone add-on. Classifier-free guidance (Ho & Salimans, 2021) got the same dial with no extra model at all. The trick: train the one diffusion model to work both with the prompt and without it, then at sampling time ask it for both predictions and combine them. That combination is the guidance knob, and it's now in essentially every text-to-image and text-to-video system.

The prerequisite is a diffusion model that can run both ways. You get it almost for free during training: with some probability, randomly drop the condition — replace the text prompt with an empty/null token — so the same network learns to denoise both when told a prompt and when told nothing. After training, one network gives you two predictions on demand: ε_cond (the noise it expects given your prompt) and ε_uncond (the noise it expects given no prompt at all).

These two predictions point in slightly different directions. The unconditional one heads toward "any plausible image." The conditional one heads toward "a plausible image that matches the prompt." The difference between them, ε_cond − ε_uncond, is therefore the conditional direction — the part of the prediction that is specifically because of the prompt. Guidance is nothing more than deciding how far to travel along that difference vector. The whole method reduces to: get both predictions, take their difference, and amplify it.

Guidance starts from the unconditional prediction and steps past the conditional one — an extrapolation, not an interpolation. That "overshoot" is what makes prompt adherence stronger than plain conditioning:

Read it as: begin at the unconditional prediction, then travel (1+w) times the conditional-direction vector. At w = 0 you land exactly on ε_cond — normal conditioning. As w grows you shoot beyond it, exaggerating the very features the prompt is responsible for, which is why higher guidance gives tighter, more literal prompt-following. But it's an extrapolation, and extrapolating too far leaves the region where the model's predictions are reliable — hence the familiar failure of high guidance scales: blown-out colors, harsh contrast, and artifacts. The whole art of setting the slider is finding how far you can push before the extrapolation breaks.

The guided noise prediction is a weighted combination of the two, with guidance weight w:

w = 0 gives ε_cond (plain conditioning); w = −1 gives ε_uncond (no prompt). For ε_cond = 2, ε_uncond = 1: at w = 3, ε = 4·2 − 3·1 = 5 — pushed well past the conditional value of 2.

The same thing, written as an extrapolation, makes the geometry explicit — start at the unconditional prediction, travel (1+w) along the conditional direction:

Identical to the first form — expand it and the ε_uncond terms recombine. The bracket ε_cond − ε_uncond is the conditional direction; (1+w) is how far you go. The runnable version below computes the guided prediction, shows the two forms agree, and sweeps the weight.

Classifier-free guidance made conditional diffusion both simpler and more controllable:

The cost is compute: each denoising step runs the model twice (conditional and unconditional), roughly doubling inference cost. And the quality-vs-adherence trade-off is real — too much guidance over-saturates and distorts. But those are prices worth paying for direct, reliable control over how literally a model follows its prompt.

Classifier-free guidance is now the default way conditional diffusion models are steered. The "guidance scale" (CFG scale) in Stable Diffusion and every mainstream text-to-image tool is exactly the w in this formula, and the same mechanism extends to text-to-video systems like Sora. A huge fraction of the practical controllability people enjoy in image generation — the ability to say "follow my prompt more" and have it work — is this one weighted subtraction.

Its ubiquity also made its main cost worth attacking. Because guidance doubles the forward passes per step, later work on guidance distillation trained models to reproduce the guided output in a single pass, folding the effect into the network so you keep the control without the 2× compute — an important optimization for fast and real-time generation. Variants also refine where guidance is applied (only at certain noise levels) or how it's scheduled across the sampling trajectory. But the core object — two predictions, extrapolate along the difference — is remarkably stable, and it's one of those rare ideas that is both a one-line formula and a cornerstone of a whole generation of creative tools.