Hierarchical MARL with PV Forecasting and Dynamic Pricing in Smart Grids

Abstract

The rapid integration of distributed energy resources and the emergence of prosumers—households that both consume and produce electricity—present significant challenges for traditional power grids. This paper proposes a Hierarchical Multi-Agent Reinforcement Learning framework for smart community energy management. The proposed system operates across three layers: (i) a machine learning-based photovoltaic (PV) generation forecasting module using an Extra Trees regressor, (ii) independent Deep Q-Network agents that learn optimal battery charge/discharge policies for prosumers, and (iii) a configurable C entral H ub t hat r edistributes s urplus PV energy from prosumers to consumers. Simulation results over a 24-hour horizon (96 time steps of 15 minutes) across a 15-home community demonstrate a 21.7% reduction in average household electricity bills compared to an uncoordinated baseline. The DQN agents exhibit stable convergence with mean squared error loss below 0.005, and the Central Hub allocation strategies enable tunable trade-offs between prosumer incentivization and consumer demand satisfaction.