Reinforcement learning-enhanced three-layer hybrid BESS-hydrogen storage in renewable microgrids with adaptive reliability thresholds and degradation awareness

Abstract

In low-carbon microgrids, hybrid battery-hydrogen storage is crucial for managing renewable energy variability. However, current optimization techniques overlook battery degradation and rely on static reliability standards that fail to adapt to changing system conditions. This paper presents a three-layer framework enhanced by reinforcement learning (RL) that dynamically adjusts reliability thresholds (risk threshold and hydrogen discharge limit) based on system states including battery state-of-charge, hydrogen level, renewable forecast, and depth-of-discharge history. The lower layer performs hourly linear programming dispatch with degradation-aware battery modeling. The middle layer uses a Q-learning agent to learn context-dependent threshold policies, and the upper layer employs an Improved Equilibrium Optimizer (IEO) for capacity planning. When tested on a renewable-dominated microgrid, the RL-adaptive configuration achieves a 23.2% reduction in battery degradation costs, 38.1% reduction in deep cycles (DoD > 60%), and extends battery lifetime by 23.5% (from 8.5 to 10.5 years) compared to fixed-threshold hybridization. Renewable curtailment decreases by an additional 17.4%, while Expected Energy Not Supplied (EENS) improves by 1.45 kWh/day (∼530 kWh/year). These significant technical and lifespan benefits demonstrate that RL-based adaptive coordination is a feasible, scalable solution for future autonomous microgrids.