DeepSeek-R1,
explained.

Rung 4 establishes that reasoning can be trained and does not say how. This rung's change is to show that the recipe can be far simpler than anyone expected and to publish it. The starkest version is the variant trained with no supervised fine-tuning at all: reinforcement learning applied directly to a base model, with a reward that checks only whether the final answer is correct and whether the output is in the required format. No human reasoning demonstrations, no step-level labels, no learned reward model to game.

What emerges under that reward is the finding. The model's responses get longer on their own over training — nothing rewards length — and behaviours the ladder below had to scaffold appear inside a single generated stream: reconsidering an approach, checking its own work, backtracking from a line that is not working. Rung 3 built that with an external search; here it is a policy the model arrived at because it raised the probability of being right.

Stated in the units this ladder compares on: rung 9 of the post-training ladder argues that supervising the process beats supervising the outcome, and buys that with an enormous annotation effort. This rung reaches comparable reasoning behaviour with outcome supervision only, paying in reinforcement-learning compute instead of in labels. The released version adds a small amount of cold-start data before the reinforcement learning, for reasons the standing block explains, and the weights and method are open.

A standard chat model answers fast — it predicts a response in one flowing pass. That's fine for "summarize this email," but it stumbles on problems that need genuine multi-step work: hard math, tricky code, layered logic. Answering such a question well requires thinking — trying an approach, checking it, backing up, trying another — before committing to a final answer.

OpenAI's o1 had shown that letting a model generate a long internal chain of thought before answering dramatically boosts reasoning — but the recipe was closed. The open question DeepSeek-R1 set out to answer: how do you actually train a model to think like that? And could you do it without a mountain of hand-written reasoning examples?

The boldest experiment came first: DeepSeek-R1-Zero. Take a base model and skip the usual supervised fine-tuning on example reasoning entirely. Instead, apply reinforcement learning directly, with a shockingly simple reward — mostly just: is the final answer correct? (checkable for math and code), plus a small reward for putting the reasoning in the right format.

Crucially, nothing rewards the reasoning itself — no human ranks the steps, no example solutions are copied. The model is only ever told whether it landed on the right answer. Everything about how to get there is left for it to discover. It's the leanest possible teaching signal: outcome only.

The result was the paper's headline. Left to maximize correctness, the model discovered — with no instruction to do so — that thinking longer helps. Over training, its responses grew from short guesses into extended chains of reasoning: laying out steps, computing intermediate results, and exploring more than one path before answering. Accuracy on hard math and coding benchmarks climbed steeply alongside this growing "thinking time."

This is a striking demonstration that a complex skill can be incentivized rather than imitated. Nobody wrote reasoning examples for R1-Zero to copy; the reasoning behavior was an emergent strategy for earning more reward. Give a capable base model the right pressure, and the sophisticated behavior falls out.

You can see the emergence fall out of the numbers. If expected accuracy rises with thinking length but each step costs a little compute, reward pressure drives the chain to exactly the length where the next step stops paying for itself:

No one wrote a single reasoning example; long chains appeared purely because they earn more reward. The runnable version below grows the chain and shows the reward is ungameable.

The most talked-about observation: during training, the model spontaneously learned to stop and reconsider. Mid-solution, it would effectively say "wait — let me rethink this," abandon a flawed approach, and try again. It also learned to spend more thinking on harder problems and less on easy ones, budgeting its own effort.

None of this self-correction was designed in. It emerged because catching your own mistakes gets more answers right, and the reward found it. The authors called it an "aha moment" — a genuinely surprising sign that reflective, self-checking behavior can arise from pure outcome-based reinforcement.

R1-Zero reasoned impressively but communicated poorly. With only correctness rewarded, its chains of thought were often hard to read and sometimes mixed languages mid-answer — nobody had rewarded clean, human-friendly output. Powerful, but not pleasant to use.

So the full DeepSeek-R1 added structure around the pure-RL core. First, fine-tune the base model on a small batch of high-quality cold-start reasoning examples to instill a clean style. Then run the large-scale RL for reasoning. Then a round of rejection sampling and more training to polish helpfulness and readability. The emergent reasoning is preserved; the output becomes coherent and well-behaved.

Running large-scale RL is expensive, so DeepSeek showed a cheaper path to spread the capability: distillation. Use the strong R1 to generate many high-quality reasoning traces, then simply fine-tune smaller open models (in the 1.5B–70B range) on those traces. No RL needed for the small models — they learn to reason by imitating R1's worked solutions.

The distilled models inherited a large share of R1's reasoning — small models that punch far above their size on math and code. It's strong evidence that reasoning skill, once created, transfers cheaply, putting capable reasoners within reach on modest hardware.

The clean RL recipe works best where answers are automatically checkable — math, code, formal logic. For open-ended tasks with no crisp "correct" signal (writing, judgment, taste), outcome-based RL is far harder to apply, so R1's edge is narrower there. Long chains of thought also cost real test-time compute: thinking longer means more tokens, more latency, more money per answer.

And reasoning fluency isn't truthfulness — a confident, well-structured chain of thought can still reach a wrong conclusion, and the visible "thinking" doesn't guarantee the model actually followed that logic. Emergent self-correction reduces errors; it doesn't eliminate them.

DeepSeek-R1 opened the reasoning era to everyone. It reproduced o1-class reasoning with an openly published recipe and open weights, and — through R1-Zero — showed something conceptually deep: advanced reasoning can be incentivized into existence by rewarding correct outcomes, not painstakingly demonstrated. That reframes how we think about teaching models hard skills.

It also cemented test-time compute — letting a model think longer at inference — as a first-class lever alongside making models bigger. Together with the cheap distillation of reasoning into small models, R1 pushed the whole field toward reasoning-first systems and made them broadly accessible, not just a frontier-lab privilege.

The admissions are unusually concrete. The pure-reinforcement-learning variant produces reasoning that is hard to read and mixes languages mid-chain — which is exactly what a reward that only checks the final answer should be expected to produce, since nothing in it ever asks the reasoning to be legible to a human. The cold-start data exists to fix that, and the paper says so. It also reports being weaker on some non-reasoning capabilities, so this is a trade rather than a free gain.

The most interesting reported negative concerns small models: distilling from the large reasoning model beat running the same reinforcement learning directly on a small one. The reasoning behaviour did not bootstrap itself at small scale — it had to be produced by something large and then transferred. That is a finding about where this capability comes from, and it echoes rung 1's threshold from the other end of the ladder.

How it held up: it was reproduced widely and quickly, which is the strongest possible contrast with rung 4's position, and it turned reasoning training from a frontier-lab capability into something many groups could run. The tension it leaves open is genuine and unresolved: outcome-only reward at sufficient scale produced what process supervision was designed to produce, and whether that means process supervision was unnecessary or that expensive reinforcement learning is rediscovering it the slow way is not settled by either result.

Answer each one out loud before you open it. Recalling an answer is what makes it stick; recognising one you have just read is not. Your score is kept on this device.

It began as a prompting trick and ended as a training objective. Read in order, the progression from "let's think step by step" to a model trained to.

1. 1Chain-of-Thought Prompting2022
2. 2Self-Consistency2022
3. 3Tree of Thoughts2023
4. 4OpenAI o12024
5. 5DeepSeek-R12025

All 110 papers, in 19 ladders →