DPO
also: direct preference optimization
Preference alignment as a simple classification loss — RLHF’s results without training a reward model.
Direct Preference Optimization skips the reward model and the RL loop: it optimizes the policy directly on preference pairs (chosen vs rejected) with a closed-form loss that provably targets the same objective. Stabler and far simpler to run than PPO-based RLHF, it became the default alignment recipe at most labs.