2026· IEEE Transactions on Automation Science and Engineering· Vol 23, pp. 14435-14449· 0 citations· 50 references
Abstract
Industrial multi-agent coordination requires distributed subsystems to collaborate under heterogeneous relationship structures whose relative importance shifts across operational contexts—physical constraints dominate startup while operational hierarchies govern steady-state. Existing multi-agent reinforcement learning approaches either ignore these structural distinctions or aggregate them uniformly, limiting adaptive coordination capabilities. This paper presents STAGE (Spatio-Temporal Aggregation via Graph Embedding), integrating multi-layer graph processing with spatio-temporal learning for context-dependent coordination. The architecture processes distinct relationship types through dedicated attention mechanisms with learned adaptive fusion, enabling coordination emphasis to adjust dynamically across operational phases. Spatio-temporal integration couples multi-layer spatial structures with multi-scale temporal dynamics through attention-based fusion mechanisms, while graph-guided hypernetworks generate mixing weights that preserve the Individual-Global-Max property essential for decentralized execution. Comprehensive evaluation on steam power plant coordination demonstrates that STAGE significantly outperforms existing multi-agent and optimization methods in both learning efficiency and asymptotic performance, while providing interpretable coordination mechanisms and maintaining the monotonicity property critical for decentralized industrial deployment. Note to Practitioners—Industrial subsystems interact through multiple relationship types whose importance varies across operational contexts. We propose a framework that separately processes these relationships and learns to adjust their emphasis adaptively. Graph-guided coordination ensures individual actions optimize system-wide performance. Evaluation on power plant configuration demonstrates superior results over conventional optimization and learning methods. The approach learns from operational data while respecting the simulator’s physical feasibility constraints. We believe this framework has strong potential for chemical processing, manufacturing, and energy management systems.
A Graph-Structured Experiential Memory framework for multi-agent coordination in dynamic manufacturing that encodes historical coordination episodes as heterogeneous relational graphs that capture task dependencies, machine states, and inter-agent collaboration patterns is proposed.
Chengxiao Dai, Zhanhui Lin, Zhaokun Yan et al.· 0 citations
Action Generation with Topology Awareness (AGTA), a topology-aware sequential decision-making framework in MARL that integrates inter-agent correlation modeling with topology-guided decision-order optimization, and outperforms the state-of-the-art counterparts.
Kun Hu, Shanghua Wen, Wendi Wu et al.· Mathematics· 0 citations
Cooperative multi-agent reinforcement learning (MARL) faces significant challenges in maintaining robust coordination under noisy observations. Although observation disturbances are often introduced independently across agents, their downstream effects on cooperative decision-making can become structured through underlying cooperation structures. We characterize this phenomenon as structured noise effects, where noise-induced decision effects exhibit local correlation among agents with stronger task-related dependencies while remaining globally heterogeneous across different agents and local structures. Existing robust MARL methods, however, rarely explicitly characterize or exploit such structure-dependent noise effects. To address this limitation, we propose SIGMA, a hierarchical collaboration framework that exploits cooperation structures to learn robust representations under noisy observations. SIGMA first organizes agents into adaptive local structures through density-based grouping and performs intra-group consensus aggregation to preserve shared task-relevant information while smoothing agent-specific representation deviations. Inter-group attention then adaptively integrates information across different groups to preserve global coordination while accommodating their heterogeneous contributions. Experiments on noisy-observation tasks in StarCraft II empirically validate the structured noise effects and demonstrate that SIGMA consistently improves robustness under observation noise while maintaining competitive performance in noise-free environments.
MANTA, a framework for Multi-Agent Network Topology Adaptation that enables communication structures to self-evolve at inference time, is introduced and shows that inference-time self-improvement can extend to the architecture of collaboration itself.
M. Huang, Jerry Wang, Yi-Cheng Lai et al.· 0 citations
This paper presents a hierarchical framework that integrates spatial-temporal graph attention network (ST-GAT) with reinforcement learning for decision and control of autonomous driving. Inspired by the principles of human cognition, the framework decomposes the driving task into two complementary levels: a high-level trajectory planning module that utilizes the soft actor-critic (SAC) algorithm within the Frenet coordinate system, and a low-level tracking control module based on the worst-case soft actor-critic (WCSAC) strategy. This hierarchical decomposition improves policy stability and sample efficiency by decoupling strategic trajectory planning from reactive control execution. Unlike previous methods, the proposed ST-GAT module enables explicit scene understanding by modeling surrounding vehicles and their interactions as a spatial-temporal graph structure. Through attention-based aggregation, the system dynamically captures road geometry and agent behaviors directly from online sensor observations, enabling mapless situational reasoning. Experimental results show that the proposed framework achieves success rates of 92.33% and 97.00% in the roundabout and five-way intersection scenarios in the CARLA simulator.
Wen-Te Hsiao, J. Liao, Yu-Chen Lin et al.· International Conference on...· 0 citations
A graph attention network-enhanced multi-agent proximal policy optimization (GAT-MAPPO) framework is proposed for cooperative guidance in adversarial engagement scenarios. A dynamic heterogeneous interaction graph is formulated over interceptors and targets at every decision epoch. Through a multi-head graph attention encoder, relational features capturing both inter-interceptor cooperation and target threat dynamics are adaptively aggregated. These graph-enriched observations are processed by a Centralized-Training, Decentralized-Execution (CTDE) MAPPO architecture, guided by a hierarchical reward function that mandates miss distance minimization, simultaneity of arrival consensus, multi-directional encirclement, and smooth control effort. Furthermore, the integration of a three-stage curriculum learning strategy allows for robust cooperative policy derivation across transitions from rectilinear to highly adaptive evasion patterns, eliminating the need for explicit rule engineering. Extensive Monte Carlo simulations confirm GAT-MAPPO’s superior performance: achieving >95% interception success rate in 4-vs.-4 scenarios and reducing mean simultaneity error by 41.4% compared to the MAPPO baseline. Comprehensive ablation and sensitivity studies validate the critical roles played by graph attention encoding, reward hierarchy design, and progressive curriculum staging.