GRPO
also: group relative policy optimization
The RL method behind DeepSeek-R1: score groups of sampled answers relative to each other, no value network.
GRPO samples a group of answers per prompt, scores them (e.g. correct/incorrect), and updates the policy toward answers that beat the group average — replacing PPO’s learned value network with a simple group baseline. Cheap enough to run at scale on verifiable tasks (math, code), it is how DeepSeek-R1 learned o1-style reasoning from a base model with no supervised warm-up.