Diffusion
LLMs.

Autoregressive (AR) models — GPT, Claude, Llama — generate token t+1 conditioned on tokens 1..t. That conditioning is the whole trick, and also the whole bottleneck: you cannot start token t+1 until token t exists. Generating N tokens means N sequential forward passes, so latency grows linearly with output length.

Batching and speculative decoding soften this, but the fundamental shape — one token per step — stays. Diffusion changes the shape.

A diffusion LLM takes a block of B positions, starts them all masked, and runs K refinement (denoising) steps. Each step looks at the whole partially-filled block and commits more tokens, until after K steps the block is clean. The key number: it emits B/K tokens per pass instead of one.

With a 64-token block refined in 8 steps, that's 8 tokens per pass — an 8× drop in pass count versus AR. Push K down and throughput climbs; the runnable below counts the passes for any B and K.

This shares a name with the image diffusion that powers Stable Diffusion and friends — but the mechanism underneath is different in a way that matters. Image diffusion adds and removes continuous Gaussian noise from pixel or latent values; "a little noise" is a real-valued perturbation of a real-valued signal.

Text is discrete. You can't add 0.3 of Gaussian noise to the token "cat" — there's no halfway between "cat" and "cot" in token space. So text diffusion uses a discrete corruption process: instead of noising values, it masks tokens and learns to un-mask them. The "noise level" is which positions are hidden, and denoising is progressive un-masking.

Same big idea — learn to reverse a corruption process, then run it from pure noise to generate — but "corruption" means something categorically different for tokens than for pixels.

The number of steps K is a dial, and it trades throughput against coherence. Fewer steps means more tokens committed simultaneously per pass — faster, but riskier: tokens chosen in the same step can't fully condition on one another, so they can come out mutually inconsistent (the classic diffusion failure where a sentence is locally fine but globally muddled). More steps restores that conditioning and the coherence with it, at the cost of speed.

The honest picture: diffusion LLMs win big where raw speed beats the last few quality points — high-throughput drafting, latency-critical UX, cheap bulk generation — while autoregressive still tends to lead on the hardest, most sequential reasoning, where every token really does depend on the one before it. It's a new point on the speed-quality frontier, not a strict upgrade.

Reported production text-diffusion systems have posted throughput figures well past a thousand tokens per second — an order of magnitude beyond typical autoregressive serving — which is exactly what you'd predict from emitting a block per handful of passes. That makes them compelling for interactive coding assistants, autocomplete at scale, and any workload where a human is watching the text stream in.

For an engineer the takeaway is a new axis to reach for: when latency dominates and the task tolerates a small quality trade, a diffusion decoder can be a step-change, not a tweak. Pair it with the image-diffusion foundations for the shared math, and with speculative decoding for the autoregressive world's answer to the same latency problem.