Self-Consistency,
explained.

Chain-of-thought prompting unlocked multi-step reasoning: show the model how to work through a problem, and it works through new ones. But standard decoding generates one chain, greedily — always the locally most likely next token. A single arithmetic slip, one misread condition, and every step after inherits the error. The chain is only as strong as its weakest link, and there's exactly one chain.

Greedy decoding has a second, quieter cost: the most probable path isn't always the most correct one. The model often "knows" better routes — they're in the distribution, just not at its peak. Greedy never visits them.

Self-consistency replaces greedy decoding with three steps. Sample a set of chains — the same CoT prompt, but with temperature, so each run takes a genuinely different reasoning route. Extract each chain's final answer, ignoring how it got there. Vote: return the answer that appears most often.

The paper frames it in one elegant phrase: the reasoning path is a latent variable. You don't care which route was taken — you marginalize the routes out and keep the answer they support. No training, no verifier, no new model. A pure decoding swap.

The magic is an asymmetry: correct answers are attractors; wrong answers are scattered. A math problem usually has many valid solution routes, and they all terminate at the same number. Errors, by contrast, are uncorrelated — a slip in step 2 of one chain and a misreading in step 4 of another land on different wrong answers. Agreement concentrates on truth; disagreement disperses over mistakes.

A bonus falls out for free: the size of the majority is a confidence signal. Twenty-eight of thirty chains agreeing means something very different from eleven — a calibration knob production systems still use.

For a change that touches nothing but decoding, the gains were startling across every model family tested (PaLM, GPT-3, LaMDA):

Accuracy climbed monotonically with the number of sampled paths — steep early gains, diminishing around 20–40 samples. That curve, accuracy as a function of inference spend, would become one of the most consequential plots in the field.

The price is explicit: N samples cost N generations. Thirty-path self-consistency is ~30× the inference bill of one greedy chain. That's why production use is selective — high-stakes answers, eval harnesses, offline pipelines — and why the sample count is a dial, not a default.

It also only works where answers are discrete and comparable: math, multiple choice, classification, code that passes tests. Open-ended generation has no clean vote — thirty essays don't have a majority. (Extensions like universal self-consistency later asked the model itself to pick the most consistent response, stretching the idea to free-form output.)

Self-consistency was the field's cleanest early proof that inference compute is a quality dial — the same model, same weights, gets measurably smarter if you let it try more. Formalize that trade, add verifiers and search, then train models to do the exploring internally, and you arrive at reasoning models. The lineage from sample-and-vote to test-time compute is direct.

It's also still the cheapest reliability upgrade in the toolbox: no retraining, model-agnostic, one loop of code. Any eval harness that reruns a flaky benchmark, any agent that generates three candidate plans and picks the consensus — that's self-consistency, doing its quiet work in production.