Chain-of-Thought
prompting, explained.

Rung 1, so the comparison is against the few-shot prompting it modifies, and the modification is almost absurdly small. Standard few-shot prompting shows the model a handful of question-and-answer pairs and then a new question. Chain of thought shows it question, worked reasoning, answer — and then the new question. Nothing else changes. No training, no gradient, no parameters, no architecture.

Why that should matter at all is the interesting part. A model answering immediately has to produce the answer in a single forward pass, so the amount of computation available is fixed by the architecture regardless of how hard the question is. Generating intermediate steps lets the model spend more forward passes on a harder problem, and lets each step condition on the written result of the last — externalising working memory into the context window, where it is exact, instead of holding it in activations, where it is not.

The second finding is the one that made the paper famous and is worth carrying up the ladder carefully. The benefit is not uniform across model sizes: below a certain scale, showing the reasoning does not help and can actively hurt, because a smaller model produces fluent chains that are wrong and then follows them. The gain appears at scale and appears sharply, which is where the vocabulary of emergent abilities attached to this result.

By 2022, big language models were fluent and knowledgeable, yet strangely bad at problems a careful ten-year-old could solve — multi-step arithmetic, word problems, chains of logic. A model prompted to answer straight away had to squeeze every step of the calculation into a single forward pass and blurt the result. On anything needing more than one step, it stumbled.

The failure wasn't knowledge. It was being forced to leap from question to answer with no room to work in between.

The authors changed the prompt, not the model. In the few examples you show a model before asking your real question — the few-shot exemplars — they didn't just include the answer. They included the reasoning that leads to it, spelled out step by step. That trail of intermediate steps is the chain of thought.

Primed with a couple of examples of what good reasoning looks like, the model imitates the format: for your new question, it writes out its own steps before committing to an answer — giving itself the scratch paper it never had.

Nothing about the model changed between these two panes — only whether it was allowed to think out loud. Breaking one hard leap into a handful of easy steps, each building on the last, is the entire trick.

There's a simple reason this compounds. If each reasoning step is right with probability p, the whole answer needs all k steps to land — p^k. Answering in one shot means each hidden step is less reliable (no room to work), so its p is lower, and p^k falls off a cliff:

The more steps a problem needs, the wider the gap — which is exactly where the paper found CoT helps most. The runnable version below makes that curve.

The paper's most striking finding: chain-of-thought is an emergent ability. On small models it barely helps, and sometimes hurts — they produce reasoning that sounds right but doesn't hold together, and the wrong steps drag the answer down with them.

Then, somewhere past roughly 100 billion parameters, the curve bends sharply upward. Suddenly the reasoning is coherent, the steps are valid, and accuracy jumps. Reasoning wasn't something you could prompt into a small model; it appeared, almost like a phase change, only once the model was big enough.

A closely related follow-up found you often don't even need worked examples. Append five words — "Let's think step by step" — to a question, and a large model starts generating its own reasoning chain unprompted. Much of the benefit of chain-of-thought, unlocked by a single sentence.

On GSM8K, a benchmark of grade-school math word problems, chain-of-thought prompting on a 540B model more than tripled accuracy versus answering directly — turning a weak result into state-of-the-art, beating a fine-tuned model with a verifier. The same pattern held across commonsense and symbolic reasoning tasks.

Same weights, same questions — the only change was letting the model show its working.

Chain-of-thought reframed inference itself: a model that can spend more steps "thinking" before answering is more capable than one forced to reply instantly. That single insight seeded everything that followed — self-consistency (sample many chains, take the majority vote), tree-of-thought, and the modern reasoning models that are explicitly trained to generate long internal chains and spend extra compute at inference on hard problems.

When today's models "think" before they answer, they're standing on this paper. It's the moment the field learned that how you let a model compute can matter as much as how big it is.

The paper is careful about something that its popular summary is not: the generated chain is not a trace of the model's computation. It is text the model produced, which influences what it produces next. Whether it describes the process that actually determined the answer is a separate empirical question, and later work on faithfulness found the two can come apart — models will produce a chain that does not reflect the features actually driving their answer, including cases where an answer is steered by something the chain never mentions. Reading a chain of thought as an explanation is therefore a mistake with a specific name, and it matters most where the stakes are highest.

The emergence framing was also contested, usefully. Schaeffer and colleagues argued that apparent sharp jumps are frequently artefacts of discontinuous metrics — score a multi-step task by exact match and smooth underlying improvement shows up as a sudden threshold, because partial credit is unavailable. That critique does not make the practical effect unreal, and it changes what it is evidence of.

How it held up: it is the substrate for everything above it on this ladder and for a large share of the post-training ladder too. The most remarkable thing about it in retrospect is the ratio — a change to the wording of a prompt, requiring no resources at all, that unlocked a capability people had assumed needed a different kind of model. That ratio is also why the rest of this ladder exists: once it was clear the capability was latent, the question became how much compute it was worth spending to extract it.

Chain-of-thought made one deceptively simple discovery: if you let a model write out intermediate steps before its answer, it gets multi-step problems right far more often. Asking for the reasoning isn't cosmetic — the tokens of "thinking" are actually where the computation happens. A model forced to answer in one step has no room to work; give it space to lay out the steps and accuracy on arithmetic, logic, and word problems jumps.

The mechanism is that a Transformer does a fixed amount of compute per token. Reasoning in the open spreads a hard problem across many tokens, so the model can build on its own partial results instead of leaping to a conclusion. "Let's think step by step" works because it literally buys more computation aimed at the problem.

It also made model reasoning inspectable. When the steps are visible you can see where it went wrong, which is invaluable for debugging prompts and for trusting an answer — though with a caveat the field later sharpened: the written chain is not guaranteed to be the true cause of the answer. It can rationalize. Treat it as a useful window, not a perfect audit trail.

The line to today is direct and enormous. Chain-of-thought was the seed of test-time compute: spend more tokens thinking to get better answers. Modern reasoning models take it to its conclusion — training the model to produce long internal chains by default, then answer. Self-consistency (vote over many chains), tree-of-thoughts (branch and search over them), and tool-augmented reasoning all build on the same core move CoT introduced: make the reasoning explicit.

Answer each one out loud before you open it. Recalling an answer is what makes it stick; recognising one you have just read is not. Your score is kept on this device.

It began as a prompting trick and ended as a training objective. Read in order, the progression from "let's think step by step" to a model trained to.

1. 1Chain-of-Thought Prompting2022
2. 2Self-Consistency2022
3. 3Tree of Thoughts2023
4. 4OpenAI o12024
5. 5DeepSeek-R12025

All 110 papers, in 19 ladders →