Physics-Guided Graph Safe Reinforcement Learning for High-Fidelity and Scalable Alternating Current Optimal Power Flow

Abstract

The alternating current (AC) optimal power flow (AC-OPF) problem is crucial for efficient power system operation, yet it remains challenging due to its nonlinear, nonconvex structure and numerous operational constraints. This paper introduces a Physics-guided Graph Safe Reinforcement Learning (PGSRL) framework that provides accurate, scalable, and physically consistent solutions to AC-OPF. The proposed PGSRL framework first models the power system as a weighted graph, encoding voltage magnitudes, load demands, and transmission line impedances to capture existing spatiotemporal correlations among power system buses. A Graph Mamba encoder with selective State Space Modeling extracts spatiotemporal correlations, which are used to define the system states within a constrained Markov Decision Process (CMDP). To solve this CMDP, we design a physics-guided safe Twin Delayed Deep Deterministic Policy Gradient (TD3) architecture containing twin critics and deterministic actors, guided by physics-informed reward functions that promote feasibility through penalty-based soft constraints. By interacting with a simulated power system environment, the introduced PGSRL learns optimal generator dispatch strategies in continuous action spaces while honoring AC-OPF constraints. Experimental results on six test benchmarks demonstrate the superiority of PGSRL compared with traditional numerical solution and supervised data-driven baselines, rather than RL methods, in terms of estimation accuracy of AC-OPF setpoints.