Self-Distillation for Supervised Fine-Tuning
Gold-Conditioned Distillation is proposed, a method that turns expert demonstrations into model-native supervision and improves new-task accuracy over SFT and generally reduces degradation on general benchmarks.