Qwen3,
explained.

Rungs 1 and 2 give you a model with one behaviour: ask, and it answers. The reasoning ladder gives you a model with a different single behaviour: ask, and it thinks at length first. Both are wrong some of the time — thinking through a factual lookup wastes tokens and latency, and answering a hard problem immediately wastes the accuracy that was available for the asking.

The change is to put both in one model and expose the choice. A thinking mode and a non-thinking mode in the same weights, with a controllable budget for how many reasoning tokens a query may spend. This is an architecture decision about products as much as about models: it means one deployment rather than two, and it means the allocation decision can be made per query by whoever knows how hard the query is.

Stated in the units the reasoning ladder established, this rung is about the slope of the test-time compute curve rather than its ceiling. Reasoning tokens buy accuracy log-linearly, so the marginal token is worth very different amounts on an easy and a hard query — and the quantity to optimise is therefore accuracy per token spent across a whole workload, not accuracy on the hardest example. Always-think and never-think are both corner solutions to that problem, and a difficulty-gated budget is the interior one.

Reasoning models like DeepSeek-R1 showed that letting a model generate a long chain-of-thought before answering dramatically improves hard problems — at the cost of many extra tokens. That created an awkward split: teams deployed a slow "reasoning" model for math and code, and a fast "chat" model for everything else, and had to route between them.

Qwen3 collapses that split. A single model supports a thinking mode (emit reasoning tokens, then answer) and a non-thinking mode (answer directly), and you choose per request. More than a toggle, it exposes a thinking budget — a cap on reasoning tokens — turning test-time compute into a continuous dial. One set of weights, tuned to be good at both, so you stop paying to host and route between two models.

The thinking budget is a cap on how many reasoning tokens the model may spend before it must commit to an answer. It makes the test-time-compute trade-off explicit and tunable: budget 0 is pure direct answering; a large budget lets the model reason at length. Two empirical facts about that curve make the budget powerful.

First, the accuracy gain from thinking saturates — the first reasoning tokens help a lot, later ones little, so past some point more thinking is wasted. Second, thinking helps far more on hard queries than easy ones: an easy question is usually answered correctly with no reasoning at all, leaving almost no headroom for thinking to add. Together these mean the value of a token of thinking depends heavily on which query it's spent on — and that's the opening for a smarter policy than "always" or "never".

If thinking mostly helps hard queries, the efficient policy is obvious: gate it on difficulty. Estimate how hard a query is, skip thinking when it's easy, and spend the budget when it's hard. That's how a controllable budget beats a fixed one — it moves compute to where the marginal token buys the most accuracy.

Concretely: on a batch of mixed queries, always-thinking pays the full reasoning cost on every request, including the majority that were easy and gained nothing. Gating on difficulty pays that cost only on the hard subset, keeping essentially the same accuracy for a fraction of the tokens. The saturation point also tells you how much to spend once you do think — enough to catch the steep part of the curve, not so much that you're past diminishing returns.

Model accuracy as a direct-answer baseline plus a saturating gain from thinking, where the headroom for that gain scales with query difficulty:

d = difficulty, t = thinking tokens, τ = saturation scale. At t=0, A=1−d (direct answer). The gain term is scaled by d, so easy queries (small d) have almost no headroom — thinking barely moves them — while hard queries (large d) gain a lot. The (1−e^^−t/τ) factor saturates: early tokens help most.

Now the policy. Gate thinking on a difficulty threshold, and only spend the budget when the gate fires:

Easy queries cost 0 reasoning tokens; hard ones cost B. Against always-thinking (B on every query), the gate keeps the accuracy that mattered while spending a fraction of the tokens — the accuracy-per-token win. The runnable version below shows thinking helping hard queries far more, and adaptive budgeting undercutting always-think on tokens.

The Qwen3 technical report comes from the Qwen team at Alibaba, released in May 2025. It covers six dense models from 0.6B to 32B parameters and two mixture-of-experts models, Qwen3-30B-A3B and Qwen3-235B-A22B, all released under the Apache 2.0 licence.

Architecture and pretraining

The models use grouped-query attention, SwiGLU, rotary positions and RMSNorm pre-normalization, with one change from Qwen2: the bias on the query, key and value projections is removed and QK-Norm is added for training stability. The MoE models have 128 experts with 8 active per token and, unlike Qwen2.5-MoE, no shared expert. Pretraining covers 36T tokens in 119 languages and dialects, in three stages: a general stage of more than 30T tokens at a 4,096-token context; a reasoning stage of about 5T higher-quality tokens weighted toward STEM, code, reasoning and synthetic data; and a long-context stage of hundreds of billions of tokens at 32,768 tokens, with the rotary base frequency raised from 10,000 to 1,000,000.

Post-training the flagship models

1. Long chain-of-thought cold start. Math, code, logic and STEM problems with verified answers or test cases. Queries that cannot be verified, or that a strong instruct model answers without reasoning, are removed, and candidate responses from QwQ-32B are filtered for wrong answers, repetition, guesswork and language mixing. The aim is to plant reasoning patterns with few samples and few steps, not to maximize scores.
2. Reasoning reinforcement learning. GRPO on 3,995 query-verifier pairs chosen to be unseen in the cold start, learnable by the cold-start model, and as hard as possible, with large batches, many rollouts per query and controlled entropy. For Qwen3-235B-A22B, AIME'24 rose from 70.1 to 85.1 over 170 RL steps.
3. Thinking-mode fusion. Continued supervised fine-tuning on a mix of thinking data, produced by rejection sampling from the stage-2 model, and non-thinking data covering coding, math, instruction following, writing and more. The chat template accepts /think and /no_think flags in the user or system message; non-thinking answers keep an empty thinking block so the format stays the same, and in a multi-turn chat the most recent flag wins.
4. General reinforcement learning. A reward system spanning more than 20 tasks, including instruction following, format following (honoring the flags and the thinking tags), preference alignment, tool calling in multi-turn agent episodes, and retrieval-augmented answers. Rewards are rule-based, model-scored against a reference answer, or taken from a preference-trained reward model where no reference exists.

The thinking budget

The budget is not a separately trained skill; the report says it emerged from thinking-mode fusion. When the thinking reaches the user's token budget, generation is halted, a fixed sentence saying time is up is inserted with the closing think tag, and the model writes its answer from the partial reasoning it has.

Strong-to-weak distillation

The four stages run only for the two flagships, Qwen3-235B-A22B and Qwen3-32B. The 0.6B to 14B dense models and the 30B-A3B MoE are distilled instead: first off-policy, on teacher outputs generated in both modes, then on-policy, where the student generates responses and is trained to match the teacher's token distributions by minimizing KL divergence.

Qwen3 sets how much to think with a template, not a learned controller. Given a prompt x, a mode set by the most recent /think or /no_think flag, and a budget B:

• zt — the thinking tokens, sampled until the model closes its own think block at step T_stop or the budget runs out
• B — the caller's cap on thinking tokens
• sstop — a fixed sentence telling the model time is up and it must answer from the thinking it has
• y — the final answer, conditioned on whatever thinking exists

When B is at least T_stop the budget never binds, and this is full thinking mode. As B shrinks, the answer is written from an ever shorter partial chain, and the report shows accuracy on math, code and science benchmarks rising smoothly as the budget grows. The /no_think extreme is not a budget of zero with a forced stop: it is the empty think block the model was trained on directly during thinking-mode fusion, which is why it behaves as a genuine direct answerer rather than a reasoner cut off mid-thought.

The flagships were evaluated in both modes against leading models of early 2025, including DeepSeek-R1 and OpenAI's o1 in thinking mode and DeepSeek-V3 and GPT-4o in non-thinking mode, across math, code, reasoning, alignment, agent and multilingual benchmarks. The report also ablates its own training stages.

The thinking-budget experiments show scores on AIME, LiveCodeBench and GPQA-Diamond climbing smoothly as the budget increases, which is what makes the budget usable as a dial rather than a switch.

The trade-offs the report admits. Stages 3 and 4 improved general abilities, instruction following and agent skills, but lowered thinking-mode scores on the hardest tasks such as AIME'24 and LiveCodeBench; the team attributes this to training on a broader range of tasks and accepted it for versatility. Distillation beat reinforcement learning for the small models: on-policy distillation from a flagship reached better scores than running RL on the same student, for about a tenth of the GPU hours. And the hybrid design itself did not last: in July 2025 the team released separate Instruct and Thinking versions of Qwen3-235B-A22B, the 2507 models, saying that training the two modes separately gave better quality than one hybrid model.

The hard part is not the two modes but the gate, and it is genuinely hard for a reason that does not go away: deciding how much thinking a question needs is a judgement about difficulty made before the question has been worked, and the cases where that judgement is wrong are exactly the cases that matter — a question that looks simple and is not. A budget dial hands that problem to the caller, which is honest, and does not solve it.

The benchmark figures are vendor-reported, and the accuracy-per-token framing makes cross-model comparison harder rather than easier, since two models can be compared at equal quality or at equal cost and those give different orderings. Worth asking of any such claim: at what budget, against which baseline, on whose workload.

How it held up: the pattern spread fast and is now ordinary. A user-visible thinking toggle, a reasoning-effort parameter, or an automatic difficulty router — the tiered routing at the top of the frontier ladder is the same idea moved server-side and hidden — are all versions of the allocation problem this rung named. That an open model shipped it as an explicit control rather than a hidden policy is the part worth keeping.

Qwen3 reframed reasoning from a property of the model to a property of the request. The reasoning-model wave asked "is this a thinking model?"; Qwen3 answers "how much should this query think?" — the more useful question once you're paying per token in production. Treating test-time compute as an adjustable budget, rather than baked-in, is now a common design point, and other frontier models have adopted similar controllable-reasoning behavior.

The underlying principle is broader than any one model: reasoning tokens are compute, compute has diminishing returns, and returns depend on the problem — so the right amount of thinking is adaptive, not fixed. Qwen3 packaged that principle into a single deployable model with a knob a developer can actually turn. The frontier isn't only about models that can think harder; it's about models that know when to, and let you decide when they don't.

Work each one out before you open it — a number, a prediction or a reason, not a definition. Producing the answer is what makes it stick. Your progress is kept on this device.

Five releases you can actually download, in the order each one answered the last. This is where "what does a frontier recipe look like" is documented in public.

1. 1Llama 22023
2. 2Llama 32024
3. 3Qwen32025
4. 4Kimi K22025
5. 5Kimi K32026

All 110 papers, in 19 ladders →