Skip to content
Conference

Curriculum-Aware Advantage Weighting and Coordination Regularization for Multi-Agent RL

Aug 2026 · 2026 IEEE 22nd International Conference on Automation Science and Engineering (CASE) · pp. 2026-2031 · 0 citations · 25 references

Abstract

Training robust policies in Multi-Agent System (MAS) remains challenging due to high sample complexity, unstable policy updates, and insufficient coordination among agents. Multi-Agent Proximal Policy Optimization (MAPPO) has recently demonstrated strong performance in cooperative multi-agent tasks; however, it does not explicitly leverage curriculum-aware optimization and lacks mechanisms that encourage consistent team-level coordination. In this paper, we propose an enhanced MAPPO framework that improves learning efficiency and cooperative behavior in complex environments. Specifically, we introduce Curriculum-Aware Advantage Weighting, a policy optimization strategy that dynamically scales policy gradients based on opponent difficulty during training. This mechanism allows agents to progressively emphasize more informative experiences from challenging scenarios while maintaining stability during early learning stages. In addition, we propose a multi-agent coordination loss that regularizes individual agent advantages toward a team-level objective, encouraging coordinated behaviors and reducing conflicting strategies among agents. This paper considers a typical multi-agent system scenario using the Google Research Football (GRF) environment, where a full 11-vs-11 match is adopted as the testbed for evaluating the proposed approach. Simulation results demonstrate that the proposed approach improves convergence speed, training stability, and final episodic returns compared with baseline MAPPO variants. These results indicate that integrating curriculum-aware policy optimization with explicit coordination regularization provides an effective framework for improving cooperative behavior and performance in large-scale multi-agent reinforcement learning tasks.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Optimizing Team Behavior via Extremum-seeking Control in Multi-agent Reinforcement Learning

In cooperative Multi-Agent Reinforcement Learning (MARL), agents must learn to coordinate their actions to achieve shared objectives. A key indicator of effective coordination is behavioral diversity, i.e., the agents'potential to enact distinct yet complementary control policies given the same set of observations. Alt...

S. Patel, Kshitij Jerath · 0 citations

Improving LLM Collaboration via Multi-Agent Preference Learning

This work instantiates MAPL using MARL from human feedback with a learned reward model and multi-agent direct preference optimization (MADPO), and shows that MAPL can improve collaboration quality and efficiency while approaching the performance of MARL with fixed, well-defined rewards.

Shuo Liu, Xinzichen Li, Tian-Le Chen et al. · 0 citations

MA-FPPO: Multi-Agent Flow-Pretrained Policy Optimization

This work proposes Multi-Agent Flow-Pretrained Policy Optimization (MA-FPPO), which uses online fine-tuning to improve the cooperative behavior of models pretrained with flow matching through new interactions with the environment.

Guo-Wei Zou, Hao-Nan Chen, Hai-Tao Wang et al. · 0 citations
Conference Sep 2026

Role-cognitive decomposition for multi-agent reinforcement learning in StarCraft II micromanagement

In complex, partially observable environments such as real-time strategy games, relying on decentralized multi-agent systems to learn efficient cooperative strategies is a challenging task. This paper proposes a Role Cognition Decomposition algorithm based on Multi-Agent Reinforcement Learning (RCD-MARL), which combine...

Zhen-Chuan Lei, Xian-Zhu Zheng, Zhi-Wei Zhang · 0 citations
#artificial intelligence Preprint Oct 2026

Flowing Faster to Coordinate: One-Step Online Multi-Agent Flow Policies

Multi-agent reinforcement learning (MARL) provides a powerful framework for learning coordinated behaviors through interactions with the environment. Developing MARL policies requires balancing expressive modeling of complex and multimodal action distributions with efficient training and execution. Generative policies,...

Zhuo-Ran Li, Yun-Zhan Li, Xun Wang et al. · 0 citations
Open access Oct 2026

Multi-Agent Path Planning Based on Coulomb’s Law and PER-MADDPG

Multi-agent path planning remains a challenging problem in deep reinforcement learning due to inefficient coordination, excessive exploration, and unstable training dynamics. Although Multi-Agent Deep Deterministic Policy Gradient (MADDPG) has been widely adopted for cooperative decision-making tasks, its performance i...

Fei-Fan Fang, Guo-Liang Jing, Qi Wang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.