D$^3$-MOPD: Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillation
D-MOPD (Dynamic Domain ScheDuling for MOPD), a zero-overhead scheduler that repurposes the per-domain reverse-KL signal already produced during training to adapt the domain mixture online to adapt the domain mixture online.
Zechen Sun, Zhi-Wei Zhang, Fei Zhao et al.
· 1 citation