Rethinking Relative Policy Differences Transfer of On-Policy Distillation
A unified view is developed in which the log-probability ratio between a proxy expert and a proxy reference defines an implicit reward for KL-regularized policy optimization, which enables any-to-any transfer across models that differ in scale, training stage or generation.