Bridging Learning and Passivity: Integral Reinforcement Learning with SPR Transformation for Multi-Agent Coordination

Abstract

This paper presents a framework that enables multi-agent systems to learn controllers independently through integral reinforcement learning while guaranteeing modular stability when connected. The framework addresses a gap: existing modular control methods require analytical models and cannot handle learned controllers, while multi-agent learning methods require network coupling during training. The proposed approach combines Integral Reinforcement Learning for independent training with a Strictly Positive Real transformation via a Proportional-Integral observer for post-learning certification. We establish necessary and sufficient conditions for the PI observer construction and prove that SPR-transformed agents achieve output synchronization, regulation to zero, and global asymptotic stability under strongly connected graphs. Unlike standard passivity-based coordination that synchronizes to arbitrary constants, the positive definiteness condition required by strict positive realness ensures regulation to the origin. We derive a performance bound showing that the network convergence rate and the stability margin depend on the graph topology, the agent dynamics, and the controller quality. The framework trades local optimality for modular guarantees: controllers optimized for isolated operation become conservative when connected, but the performance bound quantifies this cost, showing that better local controllers reduce the penalty. Simulations on 10 heterogeneous agents validate the framework across 800 Monte Carlo trials, demonstrating that IRL achieves statistical equivalence to optimal LQR despite 20% model uncertainty and unstable agents. Department of Electrical Engineering, Northeastern University, Boston, MA 02115, USA