Collision-aware cooperative multi-UAV path planning with hierarchical PPO-LSTM
TL;DR
The results indicate that separating waypoint-level strategy from recurrent local execution improves mission reliability and collision avoidance in the tested grid environments, while larger random-map benchmarks, fully controlled MAPPO/QMIX comparisons, and continuous 3-D simulation remain important future work.
Abstract
Coordinating multiple unmanned aerial vehicles (UAVs) for inspection, delivery, and search-and-rescue requires routing strategies that are both efficient and safety-aware under partial observability. Conventional optimisation methods and single-level reinforcement-learning policies often struggle when fleet size and obstacle density increase, since local collision avoidance and long-horizon route coordination must be learned within the same primitive-action policy. We reformulate multi-UAV path planning as a hierarchical reinforcement-learning problem and propose a two-tier controller in which a recurrent PPO+LSTM Manager selects coarse waypoint-level decisions, while a shared recurrent PPO+LSTM Worker executes local collision-aware movements. The Worker is pre-trained and then frozen, allowing the recurrent Manager to learn macro-level coordination over a reduced decision horizon. Under idealised deterministic grid assumptions, we show that enlarging the waypoint dictionary weakly expands the feasible macro-action set and can preserve safe alternatives when such alternatives exist. Experiments on controlled \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10\times 10$$\end{document} grid scenarios with 2–6 UAVs and 20–40 % obstacle coverage show that hierarchical PPO+LSTM achieves 90–93 % mission success and collision rates of at most 10 %, outperforming flat PPO and PPO+LSTM baselines. The evaluation further includes reward-coefficient sensitivity analysis, high-resolution path-planning visualisations, a hard bottleneck feasibility illustration, MAPPO-style diagnostic convergence curves, and rollout-loop runtime diagnostics. Statistical validation using Welch’s t-tests supports the significance of the main performance improvements. The results indicate that separating waypoint-level strategy from recurrent local execution improves mission reliability and collision avoidance in the tested grid environments, while larger random-map benchmarks, fully controlled MAPPO/QMIX comparisons, and continuous 3-D simulation remain important future work.