Dataset-Driven Exploratory Deep Reinforcement Learning Using SINDy-Based Data Augmentation from Limited Indoor Thermal Measurements

Abstract

This study examines the feasibility of combining Sparse Identification of Nonlinear Dynamical Systems (SINDy)-based data augmentation with an Actor–Critic deep reinforcement learning (DRL) model in a dataset-driven exploratory policy-learning configuration under limited-data conditions. The modeling and training dataset comprised 511 observations collected at 10-min intervals during the first seven days of an eight-day monitoring campaign; 10 conditions from the eighth day were reserved exclusively for decision evaluation. The SINDy equations were identified using 73 observations from one selected day, while the first measured observation of each day was subsequently used to initialize seven daily simulations, producing a 511-point augmented dataset. DRL models trained on the original, SINDy-augmented, and interpolation-augmented datasets were compared. Across the six modeled variables, descriptive trajectory-reconstruction R2 values obtained under the investigated data configuration ranged from 0.6446 to 0.8126. In a small descriptive comparison of 10 test conditions, the models trained on the original and SINDy-augmented datasets each showed complete-action agreement with the participant-derived recommendations in 7 of 10 cases, whereas the interpolation-augmented model achieved no complete-action agreements. These results support the feasibility of the framework under the investigated dormitory conditions. Further validation is required because the study involved a single building, a short monitoring period, one training run per dataset, and a small decision test set.