On-Policy Distillation
Distilling a teacher into a student using the student’s own sampled outputs, rather than a fixed teacher-generated dataset.
On-policy distillation trains a smaller student to imitate a teacher, but on the student’s own generated trajectories: the student samples, the teacher grades or corrects those samples, and the student learns from them. Matching the distribution the student actually produces avoids the train/deploy mismatch of copying a fixed off-policy dataset.