Simulation results show that the proposed Human-Interactive Lagrangian SAC (HI-LSAC) achieves significantly lower voltage-violation severity and reduced power losses compared with the other baseline methods.
Abstract
The growing penetration of distributed energy resources (DERs) has increased the operational variability of distribution networks, making voltage regulation increasingly challenging. Conventional deep reinforcement learning (DRL) methods exhibit unsafe exploration behavior, slow convergence, and limited reliability, which restrict their applicability in safety-critical power system settings. This paper presents a human-interactive reinforcement learning framework that enhances the safety and robustness of autonomous voltage regulation. The proposed approach integrates a Soft Actor-Critic (SAC) agent with an adaptive Lagrange constraint mechanism to enforce voltage limits, while a human-guidance module provides sensitivity-based corrections through coordinated capacitor-bank and Battery Energy Storage System (BESS) dispatch. These corrections are incorporated into a human-regularized actor loss, enabling the policy to internalize safe and interpretable control behavior. The framework is implemented in the PowerGym-OpenDSS environment using the IEEE 13-node feeder. Simulation results show that the proposed Human-Interactive Lagrangian SAC (HI-LSAC) achieves significantly lower voltage-violation severity and reduced power losses compared with the other baseline methods.
Hybrid microgrids integrating photovoltaic (PV) arrays, wind tur-binedriven PMSG units, fuel cells, and battery storage enhance sustainability but face serious power quality (PQ) challenges due to the intermittent and nonlinear behavior of renewable sources and loads. Traditional PI, PR, and hybrid intelligent controllers offer acceptable nominal performance but lack adaptability and predictive capability under rapidly varying disturbances. To overcome these limitations, this paper proposes a Deep Reinforcement Learning (DRL) frame-work based on the Twin-Delayed Deep Deterministic Policy Gradient (TD3) algorithm for real-time PQ management, where a multi-objective reward function guides optimal actions for voltage regulation, harmonic suppression, unbalance mitigation, and frequency stability. Vali-dation in a MATLAB/Simulink hybrid microgrid with nonlinear loads and renewable intermittency shows that the proposed DRL controller reduces THD from 8.42% to 2.11%, VUF from 3.9% to 0.7%, and frequency deviation from 0.42 to 0.08 Hz, while improving settling time by nearly 50%. Convergence and multi-run statistical analysis further confirm the robustness, stability, and reproducibility of the trained policy, demonstrating the effectiveness of DRL as an intelligent and scalable solution for next-generation microgrid PQ control.
Pratibha V. Hurkadli, G. A. Kumar, T. C. Manjunath· Advances in Data Science and...· 0 citations
The integration of distributed energy resources into active distribution networks presents fundamental challenges for voltage regulation, with voltage violations affecting 23% of distribution feeders under high renewable penetration scenarios. Existing approaches, notably MADDPG and PPO-based methods, address multi-agent coordination but fail to account for data privacy constraints in distributed control architectures, resulting in a 31% performance degradation under communication constraints. This work addresses this gap by introducing FedDRL-VC, a federated deep reinforcement learning framework for decentralized voltage control. FedDRL-VC employs a hierarchical aggregation mechanism to preserve data locality while enabling collaborative policy learning across network zones. A priority experience replay mechanism was designed to accelerate convergence on critical voltage events. Training was conducted on the IEEE 123-bus test feeder with realistic DER profiles over 10,000 episodes. FedDRL-VC achieved a voltage deviation of 2.1% on the benchmark, surpassing MADDPG by 56% (p < 0.001, Cohen's d = 1.84). Computational cost was reduced by 65%; convergence in 320 iterations versus 890 for TD3. Privacy preservation was maintained with zero data exchange between agents.
B. R. Reddy, Palakollu Divya, Suhail Afroz et al.· 2026 4th International Confe...· 0 citations
HVAC systems represent a major share of building energy consumption. Traditional control strategies are limited in coordinating energy-comfort tradeoffs across multiple zones simultaneously. Reinforcement learning (RL) offers adaptive, data-driven control that optimizes performance over time. However, deploying learned neural network controllers in safety-critical building systems remains challenging due to lack of formal safety guarantees. We propose a safety-certified deep RL framework for multi-zone residential HVAC control. Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) agents are trained in an EnergyPlus/Sinergym simulation to minimize energy consumption while maintaining thermal comfort. Post-training safety certification is performed on the PPO policy using Lipschitz-based forward invariance analysis, building on existing tools for the computation of Lipschitz constants for neural networks, to guarantee constraint satisfaction. Both agents are evaluated over an annual simulation cycle in an eight-zone variable refrigerant flow (VRF) testbed. The PPO agent achieves 67\% comfort violation reduction compared to rule-based control, while the SAC agent achieves 27.6\% energy savings. The PPO policy satisfies formal safety certification with a margin of $2.003^\circ$C. These results demonstrate the feasibility of combining reinforcement learning with post-training safety verification for multi-zone building control.
Oussama Ziadi, A. Rochd, S. I. Kaitouni et al.· 0 citations
This paper proposes an advanced metering infrastructure (AMI)-informed hierarchical energy management framework for coordinated operation of electric vehicles (EVs), photovoltaic (PV) systems, and battery energy storage systems (BESS) in campus microgrids. The proposed two-layer architecture integrates a soft actor–critic (SAC) deep reinforcement learning (DRL) agent in the upper layer with a receding horizon model predictive control (MPC) optimizer in the lower layer. The key novelty is an AMI-to-control pipeline that transforms historical 15 min smart-meter measurements into operational flexibility features and embeds them into a hierarchical SAC–MPC architecture, where the DRL layer provides adaptive coordination and the MPC layer enforces grid, storage, and EV-service constraints. The proposed framework using the real-world Pecan Street data (15 min resolution) of 73 homes across Austin, Texas and California (2014–2019) achieves a 53.1% cost reduction and a 25.7% peak demand reduction when compared with uncontrolled charging, and the proposed framework outperforms MPC-only (50.9%), DRL-only (−5.2%), and rule-based (5.1%) baselines. The statistically significant contributions of network-aware constraints, demand-response activation, and predictive look-ahead horizon are statistically significant (n = 10 independent runs) contributions (p = 0.001). The state representation informed by AMI offers directional cost improvement (+8.4%, p = 0.055) with 11% faster convergence of training. The zero network constraint violation is observed in all evaluation scenarios and the average MPC solve time is around 150 ms, which is much less than the 15 min sampling period. Sensitivity analyses show that the hierarchical DRL–MPC architecture remains computationally feasible across EV penetration, seasonal, and forecast-uncertainty scenarios. However, BESS provided no net economic benefit under the evaluated energy-only TOU tariff, increasing weekly cost by $15.25 and peak grid demand by 14.2 kW. Break-even analysis indicates that demand charges of approximately $9.9/kW per month are required for BESS to become cost-effective in the proxy system, highlighting that storage value depends strongly on tariff design and peak-demand objective formulation.
Mousa A. Aljabri, Mohammed O. Bahabri, Nasser A. Alakhrash et al.· Energies· 0 citations
To alleviate the severe voltage security and operational efficiency challenges brought about by the increasing penetration of distributed energy resources in active distribution networks, Volt/Var control (VVC) has become a key mechanism to stabilize node voltage and minimize power loss by coordinating reactive power injection. While multi-agent reinforcement learning (MARL) offers a promising decentralized control approach, its static reward functions are prone to creating harsh trade-offs between voltage constraint enforcement and cost-efficiency. In this paper, a hierarchical autonomous control framework featuring large language model-driven dynamic reward shaping (LLM-Driven DRS) is introduced to balance security and efficiency. The dynamic priority shifting (DPS) mechanism lies at the center of the framework and dynamically varies the reward weights through the detection of real-time grid bottlenecks. Under the LLM-Driven DRS framework, this mechanism successfully achieves a fluid transition between a Constraint-Dominant Phase for voltage stabilization and an Objective-Refinement Phase for economic optimization. Validation on a modified IEEE 33-bus system demonstrates that the proposed framework achieves Pareto superiority over conventional static weight strategies. Crucially, compared with the 1.250% static baseline, the absolute voltage violation rate is suppressed to 0.014%, mitigating long-tail risks of hardware degradation and inverter tripping, while active power losses are reduced by up to 33.76%. A robust safety margin is further confirmed by spatiotemporal analysis, which reveals an average minimum voltage margin increase of over 0.011 p.u. under severe stress conditions.
Electric vehicle (EV) powertrains involve complex interactions between speed regulation, energy consumption, regenerative braking, and battery thermal behavior. Most existing approaches address controller tuning and energy management separately, which may limit the overall system performance. This paper proposes an integrated deep reinforcement learning (DRL) strategy in which a single Twin Delayed Deep Deterministic Policy Gradient (TD3) agent simultaneously adjusts the proportional and integral gains of the speed controller (Kpv, Kiv), the torque modulation coefficient (Ks), and the regenerative braking factor (βreg). A multi-objective reward formulation is adopted to account for speed tracking performance, energy efficiency, regenerative energy recovery, battery thermal constraints, and driving comfort. The framework is implemented through a MATLAB R2022b/Simulink–Python 3.10 co-simulation environment that enables online interaction between the EV model and the learning agent. Performance is evaluated using the Worldwide Harmonized Light Vehicle Test Procedure (WLTP). Compared with a conventional fixed-gain PI controller, the approach reduces gross energy consumption by 16.2%, decreases speed tracking error by 43.7%, increases regenerative energy recovery by 21.4%, limits battery temperature rise by 30.4%, and lowers RMS jerk by 33.7%. The results indicate that jointly optimizing control and energy management variables can improve both vehicle dynamic performance and energy utilization. The methodology offers a practical framework for the development of adaptive and intelligent control systems in future electric vehicles.
Saber Hadj Abdallah, Fatma Ben Salem, Jaouhar Mouine et al.· Electronics· 0 citations