Rethinking Relative Policy Differences Transfer of On-Policy Distillation
A unified view is developed in which the log-probability ratio between a proxy expert and a proxy reference defines an implicit reward for KL-regularized policy optimization, which enables any-to-any transfer across models that differ in scale, training stage or generation.
Rong Wu, Dao-Cheng Fu, Yu Yang et al.
· 0 citations