A Graph-Structured Experiential Memory framework for multi-agent coordination in dynamic manufacturing that encodes historical coordination episodes as heterogeneous relational graphs that capture task dependencies, machine states, and inter-agent collaboration patterns is proposed.
Abstract
Dynamic manufacturing environments require multi-agent systems to coordinate effectively under frequent operational disturbances such as machine failures, urgent job arrivals, and processing time variations. Existing multi-agent reinforcement learning approaches treat each disturbance episode independently, discarding valuable coordination experience that could accelerate future adaptation. In this paper, we propose a Graph-Structured Experiential Memory (GSEM) framework for multi-agent coordination in dynamic manufacturing. The framework encodes historical coordination episodes as heterogeneous relational graphs that capture task dependencies, machine states, and inter-agent collaboration patterns. When a new disturbance occurs, a graph neural network-based retrieval mechanism identifies structurally similar past episodes, enabling experience-guided policy adaptation rather than learning from scratch. Experiments on dynamic flexible job-shop scheduling benchmarks with three disturbance types show that GSEM reduces makespan by 4.1%-10.0% and adaptation time by 33%-38% compared to the strongest memory-augmented baseline, with the advantage increasing under higher disturbance frequency. Ablation studies and cross-disturbance transfer experiments further validate the necessity of graph-structured encoding and similarity-based retrieval and demonstrate the cross-disturbance generalizability of learned coordination patterns.
MANTA, a framework for Multi-Agent Network Topology Adaptation that enables communication structures to self-evolve at inference time, is introduced and shows that inference-time self-improvement can extend to the architecture of collaboration itself.
M. Huang, Jerry Wang, Yi-Cheng Lai et al.· 0 citations
Industrial multi-agent coordination requires distributed subsystems to collaborate under heterogeneous relationship structures whose relative importance shifts across operational contexts—physical constraints dominate startup while operational hierarchies govern steady-state. Existing multi-agent reinforcement learning approaches either ignore these structural distinctions or aggregate them uniformly, limiting adaptive coordination capabilities. This paper presents STAGE (Spatio-Temporal Aggregation via Graph Embedding), integrating multi-layer graph processing with spatio-temporal learning for context-dependent coordination. The architecture processes distinct relationship types through dedicated attention mechanisms with learned adaptive fusion, enabling coordination emphasis to adjust dynamically across operational phases. Spatio-temporal integration couples multi-layer spatial structures with multi-scale temporal dynamics through attention-based fusion mechanisms, while graph-guided hypernetworks generate mixing weights that preserve the Individual-Global-Max property essential for decentralized execution. Comprehensive evaluation on steam power plant coordination demonstrates that STAGE significantly outperforms existing multi-agent and optimization methods in both learning efficiency and asymptotic performance, while providing interpretable coordination mechanisms and maintaining the monotonicity property critical for decentralized industrial deployment. Note to Practitioners—Industrial subsystems interact through multiple relationship types whose importance varies across operational contexts. We propose a framework that separately processes these relationships and learns to adjust their emphasis adaptively. Graph-guided coordination ensures individual actions optimize system-wide performance. Evaluation on power plant configuration demonstrates superior results over conventional optimization and learning methods. The approach learns from operational data while respecting the simulator’s physical feasibility constraints. We believe this framework has strong potential for chemical processing, manufacturing, and energy management systems.
This work introduces a diffusion model as a generative prior to produce high-quality global deployment plans, effectively avoiding the local-optima problem common in conventional reinforcement learning.
Jie Gao, Xingdan Wang, Zhiqing Tang et al.· Tsinghua Science and Technol...· 0 citations
This research bridges theoretical foundations of reinforcement learning and graph-based memory with autonomous agent workflows, and offers a practical, scalable reference framework for developing artificial intelligence technologies in complex, multi-step autonomous systems.
A spatiotemporal topology-aware graph reinforcement learning (STA-GRL) framework that integrates a spatiotemporal gated graph attention (STGGA) module that employs a temporal gating mechanism to dynamically prioritize tasks with rapidly decaying deadlines to address multi-agent dynamic task allocation limitations.
Xuexiu Liang, A. Siwocha, Yu Xia· Journal of Artificial Intell...· 0 citations
A novel DRL-based approach that integrates bidirectional scheduling with graph-theoretic features to effectively solve JSSP is introduced, which achieves superior performance compared to traditional heuristics and classic DRL methods, while maintaining competitive results against recent state-of-the-art approaches.