Physics-Informed Residual Reinforcement Learning for DC-Bus Voltage Regulation and VSG-Mediated Frequency Support in Grid-Connected PV-Storage Systems

Abstract

Grid-connected photovoltaic (PV)-storage systems require a well-regulated DC-side energy buffer to sustain converter operation and to support the frequency response produced by the AC-side virtual synchronous generator (VSG). Purely model-based DC-bus controllers can become conservative under changing source-load conditions, whereas full-action deep reinforcement learning (DRL) can issue aggressive commands outside the training distribution. This paper therefore proposes a physics-informed residual reinforcement learning framework in which nonsingular terminal sliding-mode active disturbance rejection control (NTSMC-ADRC) remains the primary DC-bus controller and a soft actor-critic policy learns only a bounded residual HESS-current compensation. The learned policy directly acts on the HESS reference current for DC-bus regulation; frequency support is not a second independent RL control channel but a coupled system-level response mediated by DC-side power availability and the VSG dynamics. An instantaneous projection hard-limits the commanded HESS reference current using current-capability and battery/supercapacitor state-of-charge (SOC)-dependent charge/discharge bounds, whereas DC-bus voltage and frequency limits are incorporated as soft penalty and monitoring terms. Accordingly, the proposed architecture makes a hard claim only about command-level current feasibility under the stated instantaneous model; it does not claim forward invariance of the SOC, voltage, or frequency state trajectories. Public-data-driven PV and load profiles are used to construct source-load disturbance scenarios. Compared with NTSMC-ADRC, the proposed method reduces the average maximum DC-bus voltage deviation from 4.37 V to 3.50 V and the average voltage RMSE from 1.27 V to 0.97 V. Across the NREL-PV, OPSD-EU, and ORNL-ISO test sets, the maximum voltage deviations are 2.5 V, 3.4 V, and 4.6 V, the aggregate constraint-violation rates are 0.20%, 0.40%, and 0.70%, and the frequency nadir remains at or above 49.88 Hz. Ablation results further expose the tracking–constraint trade-off: removing the projection improves voltage RMSE from 0.97 V to 0.91 V but increases the aggregate violation rate from 0.43% to 3.86%. Computational profiling gives 0.46-ms average and 0.71-ms worst-case inference time relative to the 20-ms supervisory control interval. These results support adaptive DC-bus regulation, VSG-mediated frequency support, and supervisory-layer computational feasibility, while hardware real-time operation and comprehensive voltage/frequency safety certification remain outside the demonstrated scope.