Reinforcement Learning for Optimizing Renewable Energy Utilization in Smart Grids: Recent Advances in Power Grids, Microgrids, and Building Energy Systems
A comprehensive review of RL-based control applications across RES-integrated energy domains, including power grids, microgrids, and building energy systems, identifies emerging trends and highlights dominant design patterns across power grid, microgrid, and building-level applications.
Abstract
The extensive deployment of renewable energy sources (RES) across modern energy infrastructure has introduced significant operational complexity, necessitating the development of advanced data-driven control strategies to ensure reliable and efficient system operation. Among these approaches, reinforcement learning (RL) has emerged as a promising paradigm for managing renewable generation and coordinating interconnected energy subsystems under uncertainty and dynamic operating conditions. The current paper presents a comprehensive review of RL-based control applications across RES-integrated energy domains, including power grids, microgrids, and building energy systems. The paper begins by outlining the fundamental characteristics of these smart grid energy environments along with the mathematical foundations of RL and its principal algorithmic families. A structured analysis of recent peer-reviewed studies is then conducted, with the literature systematically categorized according to the corresponding energy domain. A high number of impactful selected studies are further examined across multiple key dimensions, including RL methodologies, agent architectures, reward design, baseline control strategies, RES-integrated technologies, and control objectives. Based on this multi-dimensional evaluation, the review identifies emerging trends and highlights dominant design patterns across power grid, microgrid, and building-level applications. Finally, the observations are critically discussed and future research directions are outlined towards the development of scalable, practical, and reliable RL-based energy management solutions for next-generation smart grid systems.
Frequent network of renewable energy sources, electric cars, and distributed generation stations has changed traditional power systems into complicated smart grids. This change puts in place considerable uncertainty, non-linear and dynamic decision-making problems regarding energy management. Conventional optimization methods are usually unable to adapt effectively to the stochastic and time sensitive nature of contemporary smart grids. A recent development in machine learning has been presented as a means of solving these problems by use or Reinforcement Learning (RL), a branch of machine learning, which allows intelligent agents to acquire an optimal control policy by interacting with the environment. The paper will be a detailed report on the implementation of reinforcement learning to solve smart grid optimized energy. The framework proposed is based on the demand-side control, scheduling of energy storage and integration of renewable energy to reduce the operational cost without affecting the grid stability and reliability. The different RL paradigms such as Q-learning, Deep Q-networks (DQN) as well as Policy Gradients are discussed in their applications in the context of the smart grid. An elaborated methodology is constructed, with its system modelling, the design of state space, design of reward functions, and processes of training. The simulated experiments prove that the RL-based management strategy is much more effective in terms of minimization of costs and peak loads and its use of renewable energy sources in comparison with traditional rule-based and optimization-based strategies. The findings indicate the versatility and scability of reinforcement learning techniques in complex power system settings. The study concludes that reinforcement learning will be a highly robust and versatile solution to next-generation optimization of cyber grids with regard to data-based and autonomous, data-based grid management systems.
F. Z. Idrissi· International Journal of App...· 1 citation
The transition from centralized fossil fuel-based power systems toward decentralized smart grids with a high penetration of renewable energy sources (RES) introduces substantial challenges in monitoring, control, coordination, and management. These challenges are particularly evident at the active power grid periphery, defined in this work as the decentralized edge layer of modern power systems comprising low-voltage distribution networks, distributed energy resources (DERs), prosumers, energy storage systems, electric vehicles (EVs), and localized intelligent control entities operating near the consumer side of the grid. This review systematically examines the role of multi-agent systems (MASs) in addressing these emerging challenges. A total of 160 articles, drawn predominantly from top-tier Q1 journals and published up to March 2026, were systematically analyzed to evaluate recent methodological advances, identify persistent research gaps, and compare existing problem formulations and mathematical techniques. The review covers MAS-based applications including distributed energy management, voltage and frequency regulation, demand-side management, microgrid coordination, EV charging coordination, resilience enhancement, and cyber-physical supervisory control. The findings indicate that although MASs offer enhanced scalability, flexibility, resilience, and decentralized decision-making capabilities, existing approaches continue to face significant limitations associated with communication latency, cybersecurity vulnerabilities, interoperability constraints, heterogeneous agent dynamics, and limited real-time experimental validation. Furthermore, this review proposes six emerging research hypotheses targeting underexplored domains, presents a methodological decision flowchart for MAS implementation and selection, and discusses future research directions involving the integration of digital twins, blockchain technologies, edge intelligence, and advanced communication architectures with MAS frameworks.
Sultan Mamun, Stelios Ioannou, N. Christofides et al.· Applied Sciences· 0 citations
The increasing penetration of distributed energy resources, variable renewable generation, battery energy storage systems, electric vehicles, and power-electronic interfaces is changing the way modern smart grids and microgrids are operated, protected, and controlled. This systematic literature review follows the PRISMA 2020 framework and examines 87 original research papers, complemented by a supplementary synthesis of 18 contextual studies that provide bibliometric, historical, and conceptual perspectives on the evolution of AI in smart grids. The primary studies are organized into six thematic clusters: energy management and forecasting; cybersecurity and intrusion detection; renewable energy integration and microgrid management; fault detection, diagnosis, and grid stability; explainable and trustworthy artificial intelligence; and emerging technologies, including digital twins, blockchain, the Internet of Things, edge computing, and federated learning. The review shows that deep learning, reinforcement learning, and ensemble machine learning are increasingly used for load forecasting, demand response, converter-interfaced renewable integration, intrusion detection, and operational optimization. However, the literature remains uneven. Fault detection, converter-aware protection, and real-time stability assessment receive considerably less attention than energy management and cybersecurity, despite their importance for inverter-based resources, grid-forming converters, electric-vehicle charging systems, and battery interfacing. Four critical gaps are identified: limited cross-grid generalizability, weak validation under realistic converter and protection constraints, insufficient adversarial robustness of AI-enabled defense systems, and limited explainability in real-time safety-critical applications. The paper provides a structured taxonomy, identifies deployment barriers, and proposes research directions for trustworthy AI in power-electronic-rich smart grids and microgrids.
Reham Alsbua, M. Al-Soeidat, Ahmad A. Salah et al.· Energies· 0 citations
: The rapid deployment of distributed energy resources (DERs), including photovoltaic (PV) generation, wind turbines (WT), battery energy storage systems (BESS), and electric vehicles (EVs), is transforming modern distribution networks by introducing bidirectional power flows, voltage variations, and increased operational complexity, thereby require enhanced system resilience. This paper presents a systematic review of multi-objective optimization approaches for interconnected multi-microgrid (MMG) systems with explicit consideration of resilience, following the PRISMA 2020 guidelines. A structured literature search and screening process was conducted across major databases, including IEEE Xplore, Scopus, and ScienceDirect, covering publications from 2015 to 2026. The selected studies are synthesised based on modelling frameworks, power flow formulations, resilience metrics, and optimization strategies. The review identifies key trends, including the growing adoption of distributed coordination schemes and advanced optimization techniques to address uncertainty and scalability. However, a critical gap is observed in the integration of resilience objectives with detailed network-constrained modelling, which limits practical applicability in real-world MMG systems. Finally, key research gaps are highlighted, and future research directions are proposed to support the development of unified, scalable, and resilient optimization frameworks for high-DER MMG systems.
Theint Theint Maw, Shuai Zhou, T. T. Lie· Energy Engineering· 0 citations
The goal of this research is to evaluate energy management issues in renewable-connected smart grids, especially when the power generation produced by solar and wind fluctuates, making it difficult to maintain balanced electricity supply. There is a need for battery energy storage systems (BESS) to operate optimally in order to minimize the dependency on electrical utilities, smooth out fluctuations, and increase the amount of energy generated from renewable sources. As such, we proposed a framework using reinforcement learning (RL) to create an intelligent BESS charge-discharge schedule that can adapt to the dynamic nature of the grid. Specifically, the model uses the state of the grid (the level of renewable power produced, how much electricity is being consumed, and the battery’s state of charge) to learn the best control actions to improve BESS operation. Our simulation results indicate that using RL to optimize BESS operation will improve the efficiency of dispatching energy, increase the percentage of renewable energy used, and decrease operating costs compared to traditional ways of controlling BESS. RL shows potential for use in adapting to energy management issues in smart grids.
Akhtam Uralov, Akmaljon Aliboyev, Nargiza Nazarova et al.· EPJ Web of Conferences· 0 citations
Microgrids must efficiently manage energy under uncertainties in renewable generation and load demand to ensure reliable and cost-effective operation. This paper investigates a microgrid system that involves renewable energy through the photovoltaic system, wind system, battery energy storage, and local load requirement with a centralized energy management system. The inflexible nature of traditional rule-based and optimizationbased approaches to solving problems can often create issues in reflection to dynamic operating conditions, and reinforcement learning approaches can produce unsafe control behavior in exploration stages. To address these issues, this paper suggests a hierarchical hybrid energy management structure that will integrate rule-based supervision and a SARSA reinforcement learning controller. Supervisory layer ensures that the system is safe by ensuring that there are operational limits such as battery state of charge limits as well as power balance conditions. The learning agent on the other hand optimizes the control choices to reduce operational costs and grid energy consumption. The outcomes of the simulation indicate that the suggested approach saves more money, learns quicker, and operates a microgrid in a stable way compared to stand alone rule-based and reinforcement learning techniques. The findings demonstrate that deterministic safety rules with adaptive reinforcement learning is an effective and helpful approach to managing energy in smart microgrids.
S. Sreekanth, P. Kiran· International Conference on...· 0 citations