DRL-Based LEO Constellation Design for Regional Navigation Enhancement
TL;DR
A hybrid framework where a Double Deep Q-learning Network agent learns a policy to adaptively control the key parameters of a Particle Swarm Optimization (PSO) algorithm is proposed, validating the potential of deep reinforcement learning methods for complex aerospace optimization problems.
Abstract
Designing Low Earth Orbit (LEO) constellations for applications like Positioning, Navigation, and Timing (PNT) is a challenging multi-objective optimization challenge. Conventional metaheuristics often suffer from premature convergence due to their reliance on static adaptive rules, limiting their effectiveness in complex search space. To address this limitation, we proposed a hybrid framework where a Double Deep Q-learning Network (DDQN) agent learns a policy to adaptively control the key parameters of a Particle Swarm Optimization (PSO) algorithm. The proposed framework formulates Walker constellation optimization as an sequential parameter control problem. Based on constellation performance feedback, the DDQN controller jointly selects the inertia weight and acceleration coefficients of PSO, guiding the PSO to more effectively balance exploitation and exploration. In a regional design case for China, our algorithm demonstrated superior performance. Compared to a 120 satellites benchmark constellation, the optimized constellation achieved a 27% reduction in Geometric Dilution of Precision (GDOP), a 27.6% enhancement in navigation accuracy, and a 5% increase in coverage multiplicity. This work establishes a robust methodology for the automated and intelligent design of LEO systems, validating the potential of deep reinforcement learning methods for complex aerospace optimization problems.