One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL
The results suggest that the diversity of the training environment, not only the policy, is critical to the generalization of multi-turn RL to real-world deployment, and releases SCOPE, an open-source framework for Population Co-Training multi-agent RL.