Fast Frequency Control of VSC-Supported Low-Inertia Power Systems Using Enhanced Prioritized Reinforcement Learning
TL;DR
An application-oriented enhanced prioritized reinforcement learning strategy for fast frequency control of VSC-supported low-inertia power systems and demonstrates favorable transient regulation, empirical robustness and generalization within the investigated operating range, and low online computational burden.
Abstract
The increasing penetration of converter-interfaced renewable generation reduces power-system inertia and increases the severity of frequency deviations following active-power disturbances. To address this challenge, this paper formulates and evaluates an application-oriented enhanced prioritized reinforcement learning strategy for fast frequency control of VSC-supported low-inertia power systems. The controller integrates a twin-critic deterministic actor–critic architecture with prioritized experience replay, reward smoothing, and actor learning-rate decay. The strategy is evaluated on a modified IEEE 14-bus benchmark and compared with a local droop-control baseline, DDPG, TD3, PPO, SAC, and model predictive control (MPC). The benchmark results show favorable transient frequency-excursion suppression relative to the investigated learning-based controllers, while SAC and MPC exhibit advantages in accumulated and steady-state frequency-regulation metrics. Leave-one-out and factorial analyses further indicate complementary and metric-dependent contributions from the three training enhancements, while no clear pairwise or three-way module interaction is resolved within the present five-seed factorial assessment. The trained policy also maintains bounded responses under the investigated communication, information, and unseen operating conditions without retraining. Moreover, its 99th-percentile actor inference time is approximately 0.12 ms under a 100 ms supervisory control interval. These results demonstrate favorable transient regulation, empirical robustness and generalization within the investigated operating range, and low online computational burden.