SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation
The results support a compact, Fisher-conditioned privileged subspace for short-budget OPSD, with strict gains in 11 of the 12 model-checkpoint combinations and an exact tie at 4B step 25.
Yun-Meng Chen, Kun Wang, Pei-Han Li et al.
· 1 citation