Manoj Ghuhan Arivazhagan

Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL

This work introduces ActObs, which also supervises the observation tokens already present in each trajectory, and traces this difference to SFT: action and observation gradients rapidly become orthogonal, while action-only training leaves a large residual observation gradient and degrades environment prediction below t...