Cooperative Satellite Constellation Tracking of Multiple Maneuvering Targets Using Multiagent Reinforcement Learning

Abstract

With the growing demand for persistent surveillance of high-value targets, satellite constellations have emerged as a promising solution. However, most cooperative tracking schemes decompose the problem into separate task assignment and execution phases, which hampers continuous tracking as the number of satellites and targets increases. To address this challenge, we propose a capability-aggregated multiagent reinforcement learning framework with an end-to-end decision-making paradigm, enabling satellites to make decentralized local decisions. Each satellite is modeled as an autonomous agent and trained with a convolutional-augmented multiagent proximal policy optimization algorithm (CA-MAPPO). A dedicated state representation is designed to incorporate surrounding targets, neighboring satellites, and intrinsic dynamics, while the reward function jointly accounts for attitude accuracy, tracking performance, energy consumption, and angular velocity penalties. A high-fidelity simulation platform is also developed to assess the framework. Results on Walker constellation tracking of maneuvering targets show that CA-MAPPO extends cumulative tracking time, enhances tracking value, and reduces average control torque by more than an order of magnitude. The framework further demonstrates robustness under varying sensor fields of view, target densities, satellite numbers, and disturbance torques, highlighting its scalability and efficiency for cooperative satellite tracking missions.