Chain-of-Thought
prompting, explained.

By 2022, big language models were fluent and knowledgeable, yet strangely bad at problems a careful ten-year-old could solve — multi-step arithmetic, word problems, chains of logic. A model prompted to answer straight away had to squeeze every step of the calculation into a single forward pass and blurt the result. On anything needing more than one step, it stumbled.

The failure wasn't knowledge. It was being forced to leap from question to answer with no room to work in between.

The authors changed the prompt, not the model. In the few examples you show a model before asking your real question — the few-shot exemplars — they didn't just include the answer. They included the reasoning that leads to it, spelled out step by step. That trail of intermediate steps is the chain of thought.

Primed with a couple of examples of what good reasoning looks like, the model imitates the format: for your new question, it writes out its own steps before committing to an answer — giving itself the scratch paper it never had.

Nothing about the model changed between these two panes — only whether it was allowed to think out loud. Breaking one hard leap into a handful of easy steps, each building on the last, is the entire trick.

The paper's most striking finding: chain-of-thought is an emergent ability. On small models it barely helps, and sometimes hurts — they produce reasoning that sounds right but doesn't hold together, and the wrong steps drag the answer down with them.

Then, somewhere past roughly 100 billion parameters, the curve bends sharply upward. Suddenly the reasoning is coherent, the steps are valid, and accuracy jumps. Reasoning wasn't something you could prompt into a small model; it appeared, almost like a phase change, only once the model was big enough.

A closely related follow-up found you often don't even need worked examples. Append five words — "Let's think step by step" — to a question, and a large model starts generating its own reasoning chain unprompted. Much of the benefit of chain-of-thought, unlocked by a single sentence.

On GSM8K, a benchmark of grade-school math word problems, chain-of-thought prompting on a 540B model more than tripled accuracy versus answering directly — turning a weak result into state-of-the-art, beating a fine-tuned model with a verifier. The same pattern held across commonsense and symbolic reasoning tasks.

Same weights, same questions — the only change was letting the model show its working.

Chain-of-thought reframed inference itself: a model that can spend more steps "thinking" before answering is more capable than one forced to reply instantly. That single insight seeded everything that followed — self-consistency (sample many chains, take the majority vote), tree-of-thought, and the modern reasoning models that are explicitly trained to generate long internal chains and spend extra compute at inference on hard problems.

When today's models "think" before they answer, they're standing on this paper. It's the moment the field learned that how you let a model compute can matter as much as how big it is.