HuatuoGPT-3: RL-Only Domain Adaptation from Base Models
One-stage Policy Optimization (OnePO), which treats teacher outputs as transient guidance for policy improvement, is proposed, which combines Adaptive Objective Evolution to strengthen learning on informative low-probability teacher tokens and Teacher Retirement to discard teacher outputs once the current policy can su...