FALCON-MASAC: Formation-Aware Attention-Enhanced Leader-Guided Control-Barrier Optimization for Safe Multi-UAV Formation Navigation in Dynamic 3-D Environments
TL;DR
FALCON-MASAC is presented, a safety-integrated multi-agent reinforcement learning framework that decomposes this task into four complementary layers: a hierarchical leader-follower paradigm that pairs a pre-trained virtual leader with followers learning a distributed cooperative policy, and a bypass-side commitment coordination layer that suppresses trajectory chattering and mitigates crossing conflicts among neighboring UAVs.
Abstract
Safe cooperative navigation of unmanned aerial vehicle (UAV) formations through three-dimensional environments with dense obstacles, dynamic threats, and unstructured terrain requires jointly addressing goal-directed navigation, formation keeping, hard safety constraints, and formation-topology consistency. We present FALCON-MASAC, a safety-integrated multi-agent reinforcement learning framework that decomposes this task into four complementary layers: (1) a hierarchical leader-follower paradigm that pairs a pre-trained virtual leader with followers learning a distributed cooperative policy; (2) a dual-scale entity-risk attention encoder (DSER-AE) that structures heterogeneous observations into entity-scale and risk-scale semantic tokens and fuses them through intra-scale self-attention and inter-scale full-sequence self-attention over the concatenated tokens; (3) a safety shield built on signed distance functions and high-order control barrier functions (SDF-HOCBF) that maintains the conditional safety certificate while the robust quadratic program remains feasible and the high-order admissibility conditions hold, and switches to a bounded best-effort fallback otherwise; and (4) a bypass-side commitment coordination layer that suppresses trajectory chattering and mitigates crossing conflicts among neighboring UAVs. The safety analysis explicitly quantifies the SDF linearization error and dynamic-obstacle prediction uncertainty and describes the bounded fallback used when certified execution is unavailable. Under the centralized-training-with-decentralized-execution (CTDE) paradigm, simulation experiments show that FALCON-MASAC substantially outperforms representative baselines: it attains a 97.3% success rate (21.7 percentage points above the strongest pure-MARL baseline and 13.0 percentage points above a post-hoc CBF-RL variant), a success-conditional steady-tracking formation error of 1.524 m, and a minimum clearance of 2.732 m. The framework also generalizes well along two out-of-distribution dimensions: obstacle density and dynamic-obstacle speed.