Diffusion Models,
explained.

Generating a realistic image is fantastically hard — a million pixels that all have to agree with each other to look like a face, a street, a cat. Before diffusion, the leading approach was GANs, which pit a forger network against a detective network. Powerful, but notoriously unstable to train and prone to producing a narrow range of outputs.

Diffusion took a completely different, almost sneaky route: instead of learning to build an image in one shot, learn to clean up a messy one — a task that's far easier to train.

Start with a real image. Add a tiny bit of random Gaussian noise. Repeat, hundreds of times, each step nudging the picture a little further toward chaos, until nothing remains but pure television static. This is the forward process — and crucially, it's fixed and requires no learning at all. It's just a controlled way of destroying an image.

Why deliberately wreck a perfectly good image? Because every noised copy is a free training example: the noisy version, paired with the exact noise that was added.

Now the learning. A neural network is trained on one deceptively simple task: given a noisy image, predict the noise that was added. If it can spot the noise, it can subtract it and step back toward a cleaner picture.

The training objective is just a regression — a stable, well-behaved "predict this value" loss, nothing like the adversarial tug-of-war of GANs. That stability is a big part of why diffusion worked so well.

To generate a brand-new image, you don't start from a photo — you start from a canvas of pure random noise. Then you run the trained network over and over: it estimates the noise, removes a little, and a slightly clearer image appears. Step by step, over hundreds of iterations, coherent structure emerges from static — like a sculptor chipping a figure out of marble, except the marble is noise.

For text-to-image, the denoising is steered by a text prompt, so the picture that assembles itself matches your words. That steering is where a model like CLIP comes in — connecting language to the image being formed.

DDPM produced image quality rivaling and soon surpassing the best GANs, with a training process that was stable, simple, and covered the full diversity of the data instead of collapsing to a few looks. It traded GANs' fragility for a calm regression objective — and quickly became the dominant approach to image generation.

Every major image generator is a diffusion model. Stable Diffusion runs the process in a compressed latent space for speed; DALL·E and Midjourney use related diffusion techniques; the approach has since spread to audio, video, and even molecule and protein design. The 2020 DDPM paper is the clean formulation they all build on.

Its enduring lesson is a beautiful piece of lateral thinking: a problem too hard to solve in one leap (make an image) became easy when reframed as a tiny, repeatable step (remove a little noise). Break the impossible into a thousand possible steps, and run them backwards.