GEMS-DQN: A Global-Enhanced Multi-Agent Scheduling Deep Q-Network for Collaborative Charging Decision Optimization in Multiple UAV Systems

Abstract

To address charging-resource contention and task-allocation conflicts in multi-UAV operations supported by a single mobile charging vehicle (MCV), this paper develops GEMS-DQN (Global-Enhanced Multi-Agent Scheduling Deep Q-Network), a centralized discrete-action scheduling framework for coordinated task and charging decisions. The framework uses a joint individual–global state representation to characterize UAV energy, task urgency, spatial information, global task progress, and charging-resource utilization. A normalized system-level reward with a dynamic conflict penalty provides explicit feedback for task-assignment and charging-resource conflicts. Per-UAV Q-values are used for feasibility masking and top-k action ranking, while beam search constructs a bounded joint-action candidate set for Monte Carlo Tree Search (MCTS) under stochastic MCV motion. Experiments are conducted over 30 independent training runs. At 800 training iterations, GEMS-DQN achieves a total score of 883.7±22.4, a task completion rate of 92.1±3.4%, an average energy consumption of 10.3±0.5%, and a conflict rate of 0.091±0.018. Compared with MAPPO, the strongest modern MARL baseline evaluated, GEMS-DQN improves total score by approximately 5.6% and task completion by 7.9 percentage points, while reducing average energy consumption by 0.4 percentage points and conflict rate by 0.050. Ablation, reward-sensitivity, and scalability analyses further demonstrate the complementary effects of global information, conflict-aware learning, and bounded look-ahead search, while revealing the expected computation–performance trade-off of the centralized framework.