Reward what you can check, not what sounds good.

RLVR — reinforcement learning from verifiable rewards — trains a language model with RL, where the reward for an output comes from an automatic, objective check rather than a human or a learned model. Did the generated code pass the tests? Does the final math answer equal the known result? Is the output in the required format? Reward = 1 if yes, 0 if no. The model generates, gets scored by the checker, and is nudged toward whatever raised its score.

That’s the entire trick, and it’s powerful precisely because the reward is grounded in reality. There is no reward model to fool, no rater to charm — only a verifier that is right by construction. It is the training method behind the current generation of reasoning models: give a model checkable problems, reward correct final answers, and let it figure out how to get there.

RLHF trains a reward model to imitate human preferences, then optimizes the policy against that learned reward. It’s how models were aligned to be helpful and harmless — but it has a structural weakness: the reward model is an approximation of what humans like, and the policy will happily exploit its errors. That’s reward hacking — the model learns to produce outputs the reward model scores highly but humans don’t actually want (sycophancy, confident-sounding nonsense, gaming length or format).

The trade is subjectivity for verifiability. RLVR gives up the ability to reward “nice writing” and gains a reward that can’t be charmed. Because the signal is honest, you can crank the RL much harder without the model drifting into convincing-but-wrong territory — which is exactly what you need to push reasoning. The two are complementary: many production models use RLVR for reasoning skills and preference-based methods for tone and safety.

Mechanically, RLVR is a standard policy-optimization loop with a verifier standing in for the reward function.

A key practical detail is how the advantage is estimated. PPO uses a separate value network; DeepSeek’s GRPO (Group Relative Policy Optimization) skips it — it samples a group of responses per prompt and uses their mean reward as the baseline, which is cheaper and fits the sparse pass/fail signal well. The KL penalty to a frozen reference keeps the model from collapsing into degenerate high-reward text or forgetting its general ability while it learns to reason.

Reward design stays deliberately simple: a correctness reward (the answer is right) plus, often, a small format reward (the reasoning is delimited, the answer is where expected). Simplicity is a feature — every knob you add is a new thing the policy can hack.

The striking finding is what the model learns without being told to. In DeepSeek’s R1-Zero experiment, RLVR was applied to a base model with no supervised reasoning examples at all — pure RL on verifiable math and code. Over training, two things emerged on their own:

R1-Zero had rough edges (messy, sometimes language-mixed output), so the full R1 recipe added a small cold-start supervised phase for readability, then RLVR, then a final polish — the RL is where the reasoning ability comes from; the SFT bookends make it usable. The headline is that reasoning is not hand-crafted; it’s an emergent solution to “get the answer right.”

RLVR is bounded by one thing: you need a cheap, reliable verifier. That draws a sharp line across problem types.

Even where a verifier exists, RLVR has failure modes worth respecting. Verifier gaming: a model can pass weak tests without solving the problem (hardcoding an output, exploiting a loose checker) — your reward is only as good as your checker. Reward sparsity: on very hard problems the model almost never gets a positive reward early, so training stalls without curriculum or partial credit. And a subtle one — RLVR mostly sharpens and elicits abilities the base model already latently has, rather than teaching brand-new knowledge; it makes a model reason better, not know more.

You may never run an RLVR loop yourself, but it shapes how you use the models it produces.

• It explains reasoning models’ strengths. They’re disproportionately good at exactly the checkable tasks RLVR trains on — math, code, logic — and no better at pure recall. Route work accordingly.
• Verifiers are leverage for you too. The same generate-and-verify logic works at inference: if you can check outputs (run the tests, validate the schema), you can sample more and keep the ones that pass — RLVR’s idea applied at test time.
• It’s why open reasoning models caught up fast. RLVR is data-efficient in human labels — the checker replaces annotators — which lowered the barrier to training strong reasoners and is part of why capable open models appeared quickly.