Preprint
Aug 2026
How Much, Then Where: Credit-Conserving Action-to-Token Allocation for Multi-Turn Agent Reinforcement Learning
FACTOR is introduced, which uses checkpoint-calibrated TD residuals to assign per-action credits that telescope to the trajectory advantage, and feedback-conditioned teacher-student likelihood gaps to allocate each credit across the realized action tokens.
Li-Chao Ma, Yang Sun, Shuai Zhao et al.
· 3 citations