Dynamic Carbon-Guided HRL-MIP Dispatch for Integrated Energy Systems

Abstract

Integrated energy systems (IESs) equipped with carbon capture and power-to-gas conversion (CCS-P2G) and shared energy storage can support low-carbon operation. However, static end-of-day tiered carbon settlement provides little guidance for 15-min intraday rolling dispatch. This paper proposes an emission-progress-aware dynamic marginal carbon price that maps daily tier thresholds to intraday reference thresholds while retaining the original end-of-day settlement rule. A two-time-scale hierarchical reinforcement learning and mixed-integer programming (HRL-MIP) framework is developed. An attention-enhanced Soft Actor-Critic agent with prioritized experience replay generates hourly dispatch targets. A 15-min MIP layer then decomposes these targets into executable setpoints and corrects infeasible decisions subject to physical constraints. In a case study based on a typical winter day, coordinated CCS-P2G and shared energy storage reduce the total operating cost to 33,359.53 CNY and net system carbon emissions to 18,732.58 kg. Relative to fixed carbon-price guidance and static tiered marginal carbon-price guidance, the proposed mechanism lowers net system carbon emissions by 15.50% and 12.56%, respectively. Across three random seeds, the proposed policy achieves a late-stage reward of $-9.58~\pm ~0.19$ , outperforming five comparison algorithms. Training remains stable under the tested carbon-price variations of ±20%. Across ten repeated tests, 960 lower-level MIP calls have a mean model-construction-and-solution time of 2.75 ms and a maximum time of 6.75 ms. These results provide sufficient computational margin for 15-min rolling execution.