Towards Solar Nowcasting: Short-Term Solar Irradiance Forecasting with All-Sky Imagers and Artificial Intelligence

Abstract

The increasing share of solar photovoltaics (PV) in power grids and buildings is reshaping the energy landscape but also introducing operational challenges due to the inherent variability of solar irradiance. Rapid cloud movements can cause short-term fluctuations in PV output, making it difficult to ensure grid stability, manage power imbalances, and optimize energy use within Building Energy Management Systems (BEMS). This thesis, titled Towards Solar Nowcasting, contributes to overcoming these challenges by advancing short-termsolar forecasting techniques, with a particular emphasis on real-time, image-based forecasting also known as solar nowcasting. To this end, the research begins with a comprehensive review of solar forecasting techniques, highlighting the growing importance of Artificial Intelligence (AI) methods in capturing complex irradiance patterns across diverse time horizons, from ultra-short (1 minute) to 24 hours ahead. A particular focus is placed on the potential of neural networks and hybrid AI models, as well as the critical need for standardized datasets and benchmarking practices to ensure accurate model evaluation and performance. This review forms the foundation for the development of innovative nowcasting solutions. Building on these insights, the thesis presents a data-driven short-termsolar forecasting framework using all-sky imagers (ASIs) and deep learning. Specifically, cloud movement is tracked using optical flow models, and future sky states are generated to serve as inputs for Convolutional Neural Networks (CNNs) and Long Short-TermMemory (LSTM) networks. This hybrid approach enables accurate Global Horizontal Irradiance (GHI) predictions up to 20 minutes ahead. The developed models demonstrate significant improvements over baseline persistence methods, achieving ramp skill scores of up to 39% under sunny conditions. To address the limitations of existing methods in complex weather scenarios, the thesis further develops an innovative hybrid AI framework that combines superpixel-based cloud detection, Support Vector Machines (SVMs), CNNs, and Kalman filtering. This approach integrates high-resolution sky images, advanced computer vision techniques, and adaptive weather classification to deliver reliable GHI forecasts for horizons up to one hour. Tested on extensive datasets from the Netherlands, the method showed marked improvements in forecast accuracy, particularly under challenging conditions such as overcast or rainy skies, where conventional models often fail. Finally, the thesis translates these forecasting advancements into practical applications for congestion management, power imbalance mitigation, and building energy management. By benchmarking statistical, AI-based, and sky-imager-driven PV forecasting techniques, the study demonstrates that the integration of real-time sky image data significantly enhances short-term PV power forecasts. This, in turn, supports grid operators in implementing proactive congestion control, reduces reliance on costly balancing reserves, and enables intelligent energy management strategies within buildings, including load shifting, battery storage optimization, and increased PV self-consumption. In summary, in this thesis, advanced short-termsolar forecasting techniques have been developed to address key operational challenges arising fromthe growing integration of solar photovoltaics (PV) into modern energy systems. By combining all-sky imaging, artificial intelligence, and hybrid machine learning frameworks, this work demonstrates significant improvements in the accuracy and reliability of solar nowcasting. The proposed methodologies provide practical solutions for congestion management, power imbalance reduction, and optimized building energy management. Overall, this thesis contributes to enabling a more reliable and efficient integration of solar energy, supporting the broader goals of grid stability, energy flexibility, and the ongoing energy transition.