A value-aware extension of Multi-Agent Observation Sharing under Communication Dropout to patch communication gaps is proposed; it is referred to as Value-Aware MARO and dynamically weighting the predictor's loss function using advantage estimates derived from the underlying actor-critic architecture.
Abstract
Robust multi-agent coordination relies heavily on inter-agent communication, which is frequently disrupted by physical and environmental constraints in real-world deployments. To maintain operation during these intermittent communication failures, agents can employ internal prediction models to estimate missing shared state information. However, predictors trained with standard reconstruction objectives treat all transitions equally. In a Reinforcement Learning context, this forces the model to waste capacity learning stochastic exploration noise and the outdated dynamics of suboptimal policies. In this paper, we propose a value-aware extension of Multi-Agent Observation Sharing under Communication Dropout (MARO) to patch communication gaps; we refer to this method as Value-Aware MARO. By dynamically weighting the predictor's loss function using advantage estimates derived from the underlying actor-critic architecture, our objective explicitly couples the predictor's learning process to the policy's evolution. This formulation focuses the model's capacity on the intentional, high-return dynamics actively reinforced by the agents. We evaluate our framework on several tasks within the Multi-Agent Particle Environment under varying communication reliability levels. Experimental results demonstrate that our approach maintains performance under declining communication reliability, particularly below 40%. While our method performs comparably in tasks where the baseline already maintains high coordination, our value-aware weighting effectively prevents the performance collapse observed in the standard predictor during high-attrition scenarios. In these environments, our method achieves an average improvement in mean returns of more than 20% and reduces performance variance by a mean of 64.7% compared to the standard unweighted baseline.
Action Generation with Topology Awareness (AGTA), a topology-aware sequential decision-making framework in MARL that integrates inter-agent correlation modeling with topology-guided decision-order optimization, and outperforms the state-of-the-art counterparts.
Kun Hu, Shanghua Wen, Wendi Wu et al.· Mathematics· 0 citations
Multi-agent path planning (MAPP) under partial observability requires agents to coordinate their movements and complete tasks efficiently without access to global information. The planning space and coordination complexity grow rapidly with increasing numbers of agents, targets, and obstacles. We formulate large-scale MAPP as a partially observable networked Markov decision process. Based on this formulation, we propose a decentralized model-based Actor-Critic using the Kronecker-factored trust region (DM-ACKTR) algorithm. The algorithm integrates local model learning with ACKTR-based policy optimization in an independent learning architecture. Each agent learns a local model to predict the next observation and reward. These predictions are used to construct additional transitions for Actor and Critic updates. A neighborhood-based communication mechanism incorporates information from nearby agents into value estimation. Region partitioning reduces each agent’s effective planning space. These improvements enable DM-ACKTR to continue outperforming the baseline algorithms as the scale of the MAPP problem increases. Experiments across three training and five evaluation scenarios show that DM-ACKTR achieves the best overall performance. Among the five evaluated algorithms, it consistently obtains the highest TCR and lowest CR, improving TCR by 2.06–4.35% and reducing CR by 11.26–25.95% relative to the respective best baselines.
Ye-Min Liu, Jinhao Yang, Xiang-Yu Ma et al.· Electronics· 0 citations
Multi-Agent Systems (MAS) have emerged as a powerful paradigm for modeling complex interactions among autonomous entities in distributed environments. In Multi-Agent Reinforcement Learning (MARL), communication enables coordination but can lead to inefficient information exchange, since agents may generate redundant or non-essential messages. While prior work has focused on boosting task performance with information exchange, the existing research lacks a thorough investigation of both the appropriate definition and the optimization of communication protocols (communication topology and message). To fill this gap, we introduce a unified framework for learning multi-round communication protocols that are both effective and efficient. Within this framework, we propose three novel Communication Efficiency Metrics (CEMs) to guide and evaluate the learning process: the Information Entropy Efficiency Index (IEI) and Specialization Efficiency Index (SEI) for efficiency-augmented optimization, and the Topology Efficiency Index (TEI) for explicit evaluation. We integrate IEI and SEI as the adjusted loss functions to promote informative messaging and role specialization, while using TEI to quantify the trade-off between communication volume and task performance. Through comprehensive experiments, we demonstrate that our learned communication protocols can significantly enhance communication efficiency and achieves better cooperation performance with improved success rates.
This paper develops a multi-agent reinforcement learning-based (MARL) delegation training that enables agents to make sequential delegation decisions while minimizing the total execution cost and introduces two new frameworks for collaboration and delegation in multi-agent systems.
Ziqing Lu, Avinash Mudireddy, Sarra M. Alqahtani et al.· 0 citations
Coordinating multiple interacting units in complex engineering systems is challenging when system interactions are difficult to model, operational information is heterogeneous, and low-level actions must satisfy strict constraints. We propose an LLM-based hierarchical framework in which the LLM coordinates interacting units based on heterogeneous operational context, while task-specific controllers or optimizers generate executable and constraint-aware actions. We further introduce Continuation-Aware GRPO to capture the consequences of coordination decisions over subsequent control intervals. Rather than judging a decision only by its immediate outcome, the method also evaluates how the system evolves afterward under the current policy. We validate the framework on multi-ramp traffic control and virtual power plant (VPP) energy management, using simplified system models for training and more realistic simulators for evaluation. Across both tasks, the proposed method consistently outperforms direct task-specific control and optimization, end-to-end reinforcement learning, rule-based and RL-based hierarchical coordination, and prompting-only LLM coordinators, demonstrating the value of heterogeneous-context reasoning, hierarchical execution, and continuation-aware policy learning.
Changhong He, Jinda Gao, Xinkuan Liu et al.· 0 citations
