DoubleFree:Exploiting Exploration History for Rollout-Free Multi-Task LLMs Training
This work reuses historical trajectories for rollout-free consolidation and precomputes teacher supervision for online-teacher-free student training and analyzes when updates from historical trajectories can approximate on-policy distillation updates and how combining trajectories across exploration stages can better s...