A QMIX-driven routing protocol for dynamic Underwater Acoustic Sensor Networks

Abstract

Reliable and energy-efficient data collection remains a fundamental challenge in Underwater Acoustic Sensor Networks (UASNs) owing to limited battery capacity, high propagation delay, dynamic network topology and unreliable communication links. To address these challenges, we propose a Cooperative Multi-Agent Reinforcement Learning (CMARL)-based data routing framework for dynamic UASNs. The routing problem is modeled as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), where each relay node acts as an autonomous routing agent using locally available information. The framework adopts a Centralized Training and Decentralized Execution (CTDE) paradigm, in which cooperative routing policies are learned using the QMIX algorithm and executed through trained local Q -networks. Each agent constructs its local observation from residual energy, relative forwarding progress towards the sink, estimated link quality and neighboring node availability, while the action space consists of cooperative next-hop selection. A UASN-oriented reward function that jointly optimizes packet delivery reliability, energy efficiency and network lifetime by encouraging reliable packet forwarding and discouraging inefficient routing decisions. Extensive simulation results show that the proposed framework consistently outperforms representative routing protocols across key metrics - packet delivery rate, end-to-end delay, and network lifetime, providing an effective, scalable and intelligent routing solution for next-generation UASNs.