Speculative Decoding
also: draft model · speculative sampling
A small model drafts several tokens ahead; the big model verifies them in one pass — same output, faster.
Speculative decoding pairs a cheap draft model with the target model. The draft proposes a few tokens; the target checks them all in a single forward pass, keeping the longest correct prefix. Accepted tokens cost one big-model pass for many tokens; rejected ones fall back safely. Output distribution is provably unchanged — it is pure latency win, typically 2–3× on decode.