Recent years have witnessed the rapid adoption of reinforcement learning (RL) in large language model (LLM) post-training, with substantial gains in mathematical reasoning and code generation. In practical systems, however, policy updates and differences between rollout and training engines can make sampled responses o...
Ya-Fei Zhang, Song-Shuo Lu, Si-Cong Liao et al.· 0 citations
TCPO casts credit assignment as score-to-credit conversion and constructs turn-level advantages through reference-based comparisons and achieves the best or tied-best best-turn Pass@8 on Qwen3-4B and DeepSeek-R1-Distill-Llama-8B, reduces turns to success, and improves multi-turn agent performance.
This work introduces LEAP (Lean Environment-Feedback via Adaptive Pruning), a scalable and computationally efficient multi-turn RL framework optimized for low-level hardware accelerator alignment and proposes a Rank-Based Reward formulation, establishing a practical paradigm for low-level code RL.
Strong students can be obtained not by selecting a single better teacher, but by deliberately constructing and compressing a complementary teacher union, in an expand-then-compress framework that couples teacher construction with multi-teacher policy distillation.