Scaling Properties of Same-Family On-Policy Distillation
It is found that early OPD training dynamics uniformly exhibit a regular *useful-transfer* regime, in which held-out accuracy (the *gold score*, $G$) rises approximately linearly in $d=\sqrt{\mathrm{KL}(\pi_\theta \Vert \pi_{\mathrm{ref}})}$, the square root of token-level reverse KL divergence from the student initial...
Yun-Tai Bao, Qin-Feng Li, Guo-qing Jiang et al.
· 0 citations