Perplexity and NVIDIA propose JOLT self-distillation training method
Perplexity and NVIDIA collaboratively propose a post-training method, JOLT: it lets the same model serve as both teacher and student, and trains the teacher to learn from the rewards of its own attempts while penalizing drift away from the student's behavior. On the MATH benchmark, JOLT achieves accuracy comparable to GRPO with 6.5 times fewer training tokens; JOLT+ additionally provides the student model with outcome-based rewards, clearly surpassing GRPO on math, coding, terminal tasks, and tool-use benchmarks.
Perplexity and NVIDIA collaboratively propose a post-training method, JOLT: it lets the same model serve as both teacher and student, and trains the teacher to learn from the rewards of its own attempts while penalizing drift away from the student's behavior. On the MATH benchmark, JOLT achieves accuracy comparable to GRPO with 6.5 times fewer training tokens; JOLT+ additionally provides the student model with outcome-based rewards, clearly surpassing GRPO on math, coding, terminal tasks, and to
Matched fieldsTitle
