Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL
This work introduces ActObs, which also supervises the observation tokens already present in each trajectory, and traces this difference to SFT: action and observation gradients rapidly become orthogonal, while action-only training leaves a large residual observation gradient and degrades environment prediction below t...
Juzheng Zhang, Disha Makhija, Manoj Ghuhan Arivazhagan et al.
· 0 citations