Self-Consistency,
explained.

Rung 1 generates a single chain — usually the greedy one — and reports where it lands. That makes the answer hostage to one arithmetic slip in one step, and it ignores an obvious asymmetry in how reasoning goes wrong.

The change: sample many chains at a temperature that makes them genuinely different, then discard the reasoning and take a majority vote over the final answers. The reasoning is treated as a latent variable to be marginalised out rather than as the output — what you want is the answer the model's reasoning distribution concentrates on, not the answer one sample reached.

The premise underneath it is what makes this more than an ensembling trick: there are many valid routes to a correct answer and idiosyncratic routes to each wrong one. Correct answers therefore accumulate votes from diverse paths, while errors scatter. Stated in units the ladder can compare, this is the first clean instance on the site of a trade that dominates everything above it — spend N times the inference compute, get a double-digit accuracy gain, with the same weights and the same prompt. Reported on grade-school maths, the improvement over single-path chain of thought is around eighteen points.

Chain-of-thought prompting unlocked multi-step reasoning: show the model how to work through a problem, and it works through new ones. But standard decoding generates one chain, greedily — always the locally most likely next token. A single arithmetic slip, one misread condition, and every step after inherits the error. The chain is only as strong as its weakest link, and there's exactly one chain.

Greedy decoding has a second, quieter cost: the most probable path isn't always the most correct one. The model often "knows" better routes — they're in the distribution, just not at its peak. Greedy never visits them.

Self-consistency replaces greedy decoding with three steps. Sample a set of chains — the same CoT prompt, but with temperature, so each run takes a genuinely different reasoning route. Extract each chain's final answer, ignoring how it got there. Vote: return the answer that appears most often.

The paper frames it in one elegant phrase: the reasoning path is a latent variable. You don't care which route was taken — you marginalize the routes out and keep the answer they support. No training, no verifier, no new model. A pure decoding swap.

The magic is an asymmetry: correct answers are attractors; wrong answers are scattered. A math problem usually has many valid solution routes, and they all terminate at the same number. Errors, by contrast, are uncorrelated — a slip in step 2 of one chain and a misreading in step 4 of another land on different wrong answers. Agreement concentrates on truth; disagreement disperses over mistakes.

A bonus falls out for free: the size of the majority is a confidence signal. Twenty-eight of thirty chains agreeing means something very different from eleven — a calibration knob production systems still use.

The counterintuitive part: a single chain can be right less than half the time and the vote still wins. All that's required is that the correct answer be the plurality — more common than any single wrong answer. Spread the errors across enough distinct wrongs and that's easy:

As you add samples, the majority converges on the mode — and the mode is the truth. The runnable version below watches that curve climb.

The price is explicit: N samples cost N generations. Thirty-path self-consistency is ~30× the inference bill of one greedy chain. That's why production use is selective — high-stakes answers, eval harnesses, offline pipelines — and why the sample count is a dial, not a default.

It also only works where answers are discrete and comparable: math, multiple choice, classification, code that passes tests. Open-ended generation has no clean vote — thirty essays don't have a majority. (Extensions like universal self-consistency later asked the model itself to pick the most consistent response, stretching the idea to free-form output.)

Self-consistency was the field's cleanest early proof that inference compute is a quality dial — the same model, same weights, gets measurably smarter if you let it try more. Formalize that trade, add verifiers and search, then train models to do the exploring internally, and you arrive at reasoning models. The lineage from sample-and-vote to test-time compute is direct.

It's also still the cheapest reliability upgrade in the toolbox: no retraining, model-agnostic, one loop of code. Any eval harness that reruns a flaky benchmark, any agent that generates three candidate plans and picks the consensus — that's self-consistency, doing its quiet work in production.

Self-consistency comes from Xuezhi Wang, Jason Wei and colleagues at Google Research's Brain team, and was published at ICLR 2023. It keeps the chain-of-thought prompt exactly as it was and replaces only the decoder. Everything below happens at inference time on an unmodified model.

1. Reuse the chain-of-thought prompt. The same hand-written exemplars as the chain-of-thought paper, so any gain comes from decoding rather than from a better prompt.
2. Sample m reasoning paths. Instead of one greedy decode, draw m independent completions with temperature sampling; the main results use m = 40. Settings were set per model family: UL2 20B and LaMDA 137B at temperature 0.5 with top-k truncation to the 40 most likely tokens, PaLM 540B at temperature 0.7 with the same top-k, and GPT-3 at temperature 0.7 with no truncation.
3. Parse each path's final answer. Every completion ends in an answer line, so each path reduces to a pair (r_i, a_i): the reasoning and the answer it reached. This needs answers that can be compared for equality, such as a number or a multiple-choice letter.
4. Aggregate. Return the answer that the most paths reached. The reasoning is discarded; only the answers vote.
5. Average over runs. Because sampling is random, reported accuracies are means over 10 independent runs of 40 samples each.

The aggregation ablation

A vote is not the only way to combine paths, so the authors compared it with weighting each path by the model's own probability of generating it. Raw sequence probability shrinks with every token and so favors short chains, which is why they also tried a length-normalized version, the exponential of the mean token log-probability. A length-normalized weighted sum scored almost exactly the same as the unweighted vote. Weighted averages, which divide out how many paths agreed, did markedly worse. The paper's explanation is that the model gives its own diverse generations similar normalized probabilities, so the probabilities add little beyond the count, and the count is the signal.

The baselines it had to beat

• Sample-and-rank. Sample many paths and return the single most probable one. With the same number of samples it trails voting by a wide margin.
• Beam search. Decode the top beams and vote over them. Beams share most of their prefixes and so most of their mistakes, and voting over beams trailed voting over samples.
• Prompt ensembles. Vote over outputs from reordered exemplars or from different exemplar sets. Self-consistency beat these as well.

Two further checks matter in practice. The gain held across temperatures, top-k values and nucleus sampling, so it is not a tuning artifact. And the share of paths agreeing with the winning answer tracked accuracy: low agreement flagged questions the model was likely to get wrong, which makes the vote margin usable as an uncertainty estimate.

For one question, sample m paths and parse each into reasoning r_i and an answer a_i. Self-consistency returns the answer with the most votes:

The weighted variants replace each vote with the model's own length-normalized probability of that path:

• m — the number of sampled paths, 40 in the paper's main results
• 1[ai = a] — 1 if path i reached answer a, else 0; this is why answers must be comparable for equality
• tk, K — the tokens of path i and its length; dividing by K stops long chains being penalized for their length
• argmaxa — the plurality answer, which does not need a majority

At m = 1 the rule is one sampled chain with no protection against a slip. As m grows, the vote converges on the mode of the model's answer distribution: correct when that mode is the right answer, confidently wrong when the model is systematically biased. Summing the weights w_i instead of counting gave nearly the same accuracy; averaging them, which throws away how many paths agreed, did worse.

The evaluation covered UL2 20B, LaMDA 137B, PaLM 540B and GPT-3 (the Codex code-davinci models) on arithmetic benchmarks (GSM8K, SVAMP, AQuA, ASDiv, AddSub, MultiArith) and commonsense ones (CommonsenseQA, StrategyQA, ARC). The baseline is the chain-of-thought paper's own method, the same prompt decoded greedily. Self-consistency used 40 samples, averaged over 10 runs.

On GSM8K, PaLM 540B went from 56.5% with greedy chain of thought, the self-consistency paper's own baseline run, to 74.4%. Every model improved, and accuracy rose with the number of sampled paths, steeply at first and flattening toward 40.

What did not work, or worked less. Weighting votes by length-normalized probability matched plain counting, and averaging probabilities did worse, so the model's confidence added nothing the count did not. Sample-and-rank and beam search both trailed voting over diverse samples, which isolates diversity as the ingredient that matters. The gains were smallest where greedy decoding was already near the ceiling, as on the easier arithmetic sets for the largest models. And the method's precondition limits where these numbers apply: every benchmark here has a short answer that can be checked for equality, which open-ended generation does not.

Two limits are structural. The cost is linear in the number of samples with no early exit, and the method needs answers that can be compared for equality — you can majority-vote a number or a multiple-choice letter, and you cannot majority-vote an essay, a proof or a piece of code without first solving the equality problem that made it hard.

The third limit is the one worth remembering, because it is a failure of the premise rather than of the method. Voting helps when errors are idiosyncratic. When a model is systematically wrong — it has learned a misconception, or the question contains a misleading cue — the samples agree on the wrong answer, and self-consistency converts a wrong answer into a confidently wrong answer. Agreement among samples measures the sharpness of the model's distribution, and people read it as confidence in the answer's truth. Those coincide only when the model is unbiased.

How it held up: as a default evaluation-time technique and as the conceptual seed of test-time compute. The idea that accuracy is something you can buy at inference rather than something fixed by the weights starts here, and rung 4 is that idea taken seriously enough to reorganise a model around. The step from here to the rest of the ladder is the recognition that if you are going to sample many candidates, choosing among them well is worth more than counting them — which is where a verifier beats a vote.

Work each one out before you open it — a number, a prediction or a reason, not a definition. Producing the answer is what makes it stick. Your progress is kept on this device.

It began as a prompting trick and ended as a training objective. Read in order, the progression from "let's think step by step" to a model trained to.

1. 1Chain-of-Thought Prompting2022
2. 2Self-Consistency2022
3. 3Tree of Thoughts2023
4. 4OpenAI o12024
5. 5DeepSeek-R12025

All 110 papers, in 19 ladders →