Block Diffusion
A hybrid that runs autoregressively across blocks of tokens but uses diffusion to fill in each block.
Block diffusion splits a sequence into blocks, generating blocks left to right (autoregressive) while denoising all tokens within a block in parallel (diffusion). It aims to keep the quality and KV-cache friendliness of autoregression while recovering some of diffusion’s parallel speed inside each block.
Worked example: a hybrid that splits a sequence into blocks, generating blocks left-to-right (autoregressive across blocks) while diffusing all tokens WITHIN a block in parallel — aiming for diffusion’s intra-block speed plus autoregression’s ability to condition on finished context and support arbitrary length. Gotcha: it tries to keep the strengths of both but adds the complexity of both (block size becomes a quality/speed knob); it is an active research direction rather than a settled architecture.