Skip to content
Open access

Hierarchical Multi-Agent Reinforcement Learning for Networked Multi-AUV Data Collection in UWSNs

2026 · IEEE Transactions on Cognitive Communications and Networking · Vol 12, pp. 12256-12267 · 0 citations · 39 references

Abstract

Underwater wireless sensor networks form the foundation of the marine Internet of Things, but timely data delivery remains challenging in deep and remote deployments. Although AUV-assisted data collection reduces reliance on energy-constrained multi-hop acoustic relays, slow vehicle mobility and repeated surfacing for data upload still degrade the Value of Information (VoI). To address this bottleneck, we propose a networked multi-AUV data-collection framework supporting opportunistic inter-AUV acoustic relaying. Buffered data can be forwarded to near-surface peers acting as mobile gateways, reducing redundant surfacing and improving delivered VoI at the surface base station. We formulate a VoI-driven cooperative control problem jointly considering sensor-specific acquisition, peer-specific relaying, uploading, idling, and continuous three-dimensional trajectory control under communication-feasibility and collision-avoidance constraints. To learn a tractable policy for this strongly coupled hybrid decision problem, we develop a hierarchical multi-agent reinforcement learning framework. The upper layer uses QMIX-style value decomposition to learn coordinated discrete task decisions from local observation histories, while the lower layer uses MADDPG with centralized critics and decentralized actors to learn task-conditioned continuous three-dimensional motion policies. The two layers are coupled through discrete-action embedding, a shared team reward, and the post-resolution environment transition, avoiding direct optimization over the full joint hybrid action space. Extensive simulations demonstrate competitive delivered-VoI performance and faster, more stable convergence than representative baselines, while substantially reducing surfacing frequency of deep-water collectors.

Read PDF

Similar papers

Preprint Aug 2026

Multi-AUV Ad-hoc network-based Target Tracking: A Value Gradient Guidance Multi-Agent Diffusion Reinforcement Learning Approach

Experimental results show that VGG-MADiffRL consistently achieves faster convergence, higher tracking accuracy, and smoother training dynamics in cooperative tracking scenarios, validating its effectiveness and practical engineering value in dynamic underwater settings.

Jiaao Ma, Chuan Lin, Guang-Jie Han et al. · 0 citations
Open access Sep 2026

Coverage Control for Underwater Acoustic Mobile Sensor Networks with Connectivity and Energy Constraints Based on Multi-Agent Deep Reinforcement Learning

Results validate that the proposed hierarchical architecture effectively mitigates the exploration burden, prevents topology fragmentation, and enables robust, energy-efficient cooperative coverage in underwater acoustic mobile sensor networks.

Jun-Feng Liu, Ming-Ru Dong, Yong-Tao Hu et al. · 0 citations
Open access Sep 2026

Joint Trajectory Design and Resource Allocation for QoS-Aware Emergency Data Collection in UAV-Assisted WPCNs: A Hierarchical DRL Approach

This paper investigates an energy efficiency maximization problem in a UAV-assisted WPCN by jointly optimizing the UAV trajectory, hovering altitude, and hybrid TDMA/NOMA resource allocation and proposes a deep reinforcement learning-driven Hierarchical Energy-Efficient Data Collection scheme, which is named DRL-HEEC.

Si-Liang Gong, Kai-Yang Qu, Qi-Sen Wang et al. · 0 citations
Sep 2026

AoI-Driven Hierarchical Learning for Cooperative Resource Sharing in Multi-Operator UAV Networks

Uncrewed aerial vehicle (UAV)-assisted networks provide a versatile paradigm for on-demand connectivity. However, in multi-operator aerial networks (MOANs), the joint optimization of cooperative resource sharing and 3D trajectory control to maintain information freshness is a complex combinatorial problem, which can be...

Atefeh Hajijamali Arani, M. Shirvanimoghaddam, A. Mehbodniya et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.