Impact of Architectural Heterogeneity and Reward Design on Coordination in IPPO-Based Multi-Agent Systems for Sequential SAR Tasks
TL;DR
This study evaluates how architectural heterogeneity and reward design influence coordination in decentralized IPPO-based Multi-Agent Reinforcement Learning (MARL) systems operating in Search-and-Rescue environments with strict sequential task dependencies and provides empirically grounded design recommendations for cooperative multi-agent systems.
Abstract
This study evaluates how architectural heterogeneity and reward design influence coordination in decentralized IPPO-based Multi-Agent Reinforcement Learning (MARL) systems operating in Search-and-Rescue (SAR) environments with strict sequential task dependencies. Controlled experiments compare homogeneous and heterogeneous teams across multiple reward structures, observation ranges, task complexities, and asymmetric role distributions in a partially observable, communication-denied, decentralized environment. The results show that homogeneous teams consistently achieve faster convergence, higher coordination efficiency, and more stable cooperative behavior than balanced heterogeneous configurations. Under hybrid global–local reward schemes, homogeneous teams achieve a higher success rate than local-only reward schemes. A dedicated test confirms this advantage stems from team composition itself, not parameter sharing: it persists even under independent, non-shared policy networks. Additional experiments suggest that asymmetric role allocation may influence coordination dynamics in heterogeneous teams. The study provides empirically grounded design recommendations for cooperative multi-agent systems, derived from controlled simulation experiments under communication-limited sequential coordination constraints.