Aug 2026· Entropy· Vol 28· 0 citations· 36 references
Medicine
TL;DR
IB-CEMARL is proposed, an information-bottleneck-guided, communication-efficient multi-agent reinforcement learning framework for UAV swarms that achieves superior cooperative performance, reduced message redundancy, and stronger robustness compared with representative communication-aware MARL baselines.
Abstract
Multi-agent reinforcement learning has emerged as a promising paradigm for cooperative unmanned aerial vehicle (UAV) swarm coordination. However, existing communication-aware MARL methods primarily focus on communication topology, message routing, and message aggregation, while the information content of the exchanged messages is often only implicitly controlled. In realistic UAV networks, inter-agent communication is constrained by limited bandwidth, communication range, energy consumption, and packet loss. It is therefore desirable for each UAV to transmit compact and task-relevant information rather than dense and redundant latent features. In this paper, we propose IB-CEMARL, an information-bottleneck-guided, communication-efficient multi-agent reinforcement learning framework for UAV swarms. We formulate inter-UAV communication as a minimal sufficient message-learning problem in which each UAV encodes its local observation into a stochastic bottleneck message before exchanging information with its neighbors. Cauchy–Schwarz divergence-based quadratic mutual information is adopted as a unified dependence measure to jointly regularize message compression, preserve decision-relevant information, and reduce statistical redundancy among neighboring UAV messages. Extensive experiments demonstrate that IB-CEMARL achieves superior cooperative performance, reduced message redundancy, and stronger robustness compared with representative communication-aware MARL baselines. In particular, IB-CEMARL improves the average return by 4.9% and reduces inter-message dependence by 29.0% compared with the KL-IB-MARL baseline while maintaining efficient communication under constrained bandwidth settings.
Unmanned aerial vehicles (UAVs) have emerged as a key enabler of next-generation Internet of Things (IoT) ecosystems, offering flexible aerial relaying to extend connectivity across dynamic vehicular ad hoc networks (VANETs) in smart city environments. However, conventional centralized approaches for UAV trajectory planning require continuous global network state aggregation, making them impractical under bandwidth and energy constraints typical of dense urban deployments. In this article, we present TRUAV, a distributed multi-agent reinforcement learning framework based on independent tabular Q-learning for joint UAV trajectory planning and routing enhancement in UAV-aided VANETs. Each UAV is equipped with a local Q-learning agent that operates purely on locally observable information, including vehicle density, packet queue states, and neighbor UAV positions, thereby eliminating the need for global state exchange. A potential-game-inspired reward design encourages spatial diversity and routing-aware UAV positioning among interacting agents while accounting for energy consumption. Numerical simulations over a large urban area with 200 mobile vehicles show that the proposed TRUAV framework achieves network coverage and packet delivery ratios comparable to centralized deep reinforcement learning methods, while also improving relay delay and energy efficiency. Finally, we discuss emerging challenges and future research directions for distributed multi-agent UAV-assisted IoT systems.
Muhammad Umar Farooq Qaisar, Lin Zhang, Zhen Chen et al.· 0 citations
Multi-Agent Systems (MAS) have emerged as a powerful paradigm for modeling complex interactions among autonomous entities in distributed environments. In Multi-Agent Reinforcement Learning (MARL), communication enables coordination but can lead to inefficient information exchange, since agents may generate redundant or non-essential messages. While prior work has focused on boosting task performance with information exchange, the existing research lacks a thorough investigation of both the appropriate definition and the optimization of communication protocols (communication topology and message). To fill this gap, we introduce a unified framework for learning multi-round communication protocols that are both effective and efficient. Within this framework, we propose three novel Communication Efficiency Metrics (CEMs) to guide and evaluate the learning process: the Information Entropy Efficiency Index (IEI) and Specialization Efficiency Index (SEI) for efficiency-augmented optimization, and the Topology Efficiency Index (TEI) for explicit evaluation. We integrate IEI and SEI as the adjusted loss functions to promote informative messaging and role specialization, while using TEI to quantify the trade-off between communication volume and task performance. Through comprehensive experiments, we demonstrate that our learned communication protocols can significantly enhance communication efficiency and achieves better cooperation performance with improved success rates.
Reliable link maintenance is currently a critical bottleneck for unmanned aerial vehicle (UAV) swarm communications in complex electromagnetic environments where UAVs encounter both external malicious jamming and internal interference. Most recent studies have treated trajectory design and resource scheduling as decoupled problems or employed standard deep reinforcement learning methods to handle static spectral scenarios. However, these approaches lead to frequent link breakages and slow convergence when dealing with dynamic topologies and spatiotemporal interference. To tackle this challenge, we proposes a joint spatial-spectral adaptive coordination (JSSAC) framework and a deep recurrent attentionbased Q-network (DARQN) approach, utilizing a multi-head attention mechanism to intelligently aggregate heterogeneous neighbor features, thereby enhancing the swarm's adaptability to dynamic network topology. Moreover, considering that the spatial distribution of drones fundamentally determines the upper bound of the signal quality, we designed a communicationaware potential field mechanism that incorporates real-time signal-to-interference-plus-noise ratio feedback. Simulation results demonstrate that compared to DQN and DRQN algorithms, the proposed algorithm achieves transmission success rates of over 92%, representing improvements of 17% and 8% respectively, while also accelerating convergence speed.
Miao Liu, Nan Qi, Hua Jiang et al.· International Mediterranean...· 0 citations
In satellite-UAV assisted communication networks, jointly optimizing the UAV’s trajectory and the multi-agent scheduling decisions to minimize the age of information (AoI) is a notoriously challenging problem. The complexity is compounded by the fundamental heterogeneity between the satellite and UAV agents, including their disparate action spaces, partial observations, and differing energy-consumption and communication-cost penalties. To address this, we formulate the problem as a decentralized partially observable Markov decision process (Dec-POMDP) and propose a novel heterogeneous multi-agent compound-action proximal policy optimization (HMACPPO) algorithm. HMACPPO leverages a centralized training with decentralized execution (CTDE) framework, using role-specific decentralized actors together with agent-specific centralized critics conditioned on the global state. Specifically, the UAV employs a compound PPO (CPPO) actor for its hybrid action space, while the satellite uses a PPO actor for discrete scheduling. Extensive simulations show that HMACPPO outperforms the compared baselines, and that the resulting coordinated policy effectively manages the trade-off between AoI, UAV energy consumption, and operational cost.
Weijie Zhou, Mengjie Yi, Yan Zhang et al.· IEEE Transactions on Cogniti...· 0 citations
Rapid advances in drone technology, combined with the growing congestion of terrestrial transport networks, are driving the emergence of the low-altitude economy. Uncrewed Aerial Vehicles (UAVs) are increasingly deployed for low-altitude economy applications such as urban logistics and transportation, yet their expansion is constrained by the scarcity of spectrum resources. Although Multi-Agent Reinforcement Learning (MARL) offers a promising decentralized approach to improve spectral efficiency of UAVs, existing MARL methods suffer from high training costs, often requiring extensive environmental interactions. To overcome these limitations, we propose a novel Multi-Agent Model-Based reinforcement learning algorithm for decentralized spectrum sharing among UAVs in the low-altitude economy, which we denote as MAMBA-UAV. Adopting a Centralized Training with Decentralized Execution (CTDE) paradigm, MAMBA-UAV equips each UAV with a learned world model that captures compact environmental representations and predicts system dynamics. These world models are then utilized during MARL training to simulate interactions, thereby reducing the reliance on repeated real-environment rollouts. Through comprehensive simulations, we demonstrate that MAMBA-UAV substantially reduces the number of environmental interactions required for UAVs to achieve competitive spectrum-sharing performance, lowering training costs while maintaining high performance.
Tianle Li, Peixi Peng, Qingyu Liu et al.· IEEE Transactions on Cogniti...· 0 citations
Search-and-rescue (SAR) operations in disaster environments require drone swarms to coordinate efficiently despite incomplete information and potential communication failures. Existing stigmergy-based approaches provide low-bandwidth coordination but rely on fixed rules, whereas multi-agent reinforcement learning (MARL) can learn adaptive behaviors but often struggles with coordination under partial observability. To address these limitations, this paper proposes a Hybrid stigmergy–MARL framework that introduces directional pheromone-gradient observations, enabling each drone to infer the direction of likely victims and unexplored regions using locally available information. The proposed framework combines reinforcement learning with four virtual pheromone layers representing coverage history, victim likelihood, environmental risk, and communication quality. Victim detection is modeled through an abstract short-range thermal/visual sensing mechanism, while environmental information is shared through pheromone-based environmental memory to reduce dependence on direct communication. The simulated environment consists of a 40 × 40 grid, where each grid cell represents a discrete two-dimensional location. Victims occupy a single grid cell, and obstacles are modeled as static two-dimensional impassable cells. Experimental results show that the proposed approach achieved 98.9% area coverage and 93.3% victim detection, compared with 81.8% coverage and 71.7% victim detection for the RL-only baseline. Ablation experiments confirmed that directional gradient observations are the primary contributor to these improvements, while communication-loss experiments demonstrated robust performance even under complete communication outage. These findings indicate that directional pheromone-gradient observations provide an effective and communication-efficient mechanism for decentralized swarm coordination, improving search effectiveness and operational robustness in post-disaster SAR scenarios.
Peter Yacoub, Mohamed Malek Kaouach, Esraa Khatab et al.· Drones· 0 citations