GR-MAPPO: A Graph-Enhanced Reinforcement Learning Framework for Narrow-Beam Directional Neighbor Discovery in UAV Networks

Abstract

Directional communication has emerged as a key enabling technology for next-generation uncrewed aerial vehicle (UAV) networks to achieve extended transmission ranges and high spectral efficiency. However, the consequent reliance on extremely narrow beamwidths imposes stringent spatial constraints that result in highly restricted partial observability. Under such severe constraints, existing neighbor discovery paradigms face significant limitations. Conventional rule-based algorithms rely on rigid, predefined scanning patterns that fail to adapt to dynamic topologies and high-interference environments, leading to inefficient discovery. While existing reinforcement learning methods offer adaptability, they struggle to cope with the extreme partial observability induced by narrow beams. These algorithms often fall into local optima, failing to leverage global perspectives and spatio-temporal contexts to derive effective strategies from fragmented observations. To bridge this gap, we propose a spatio-temporal data-driven framework tailored for directional UAV networks. Architecturally, we design a dual-stream actor network that captures temporal dependencies from variable-length sequences to mitigate local observational ambiguity through historical memory. Furthermore, we introduce an edge-aware graph critic network based on the message-passing neural network (MPNN) to aggregate network connectivity information for robust value estimation. By integrating these designs into the multiagent proximal policy optimization (MAPPO) approach, we propose the graph-enhanced recurrent MAPPO (GR-MAPPO) algorithm. Simulation results demonstrate that the proposed method significantly outperforms existing baselines in discovery efficiency and maintains robust performance under varying beam constraints and network scales.