Jun 2026· International Conference on Mobile Data Management· pp. 434-436· 0 citations· 6 references
Abstract
This manuscript summarizes ongoing doctoral research on control of building heating, ventilation, and air conditioning (HVAC) systems through deep reinforcement learning trained on data-driven simulators. The thesis investigates whether multivariate time-series forecasting models can act as reliable surrogates of physics-based building emulators for controller development. The work is motivated by the high modeling effort required by conventional model predictive control and by the practical impossibility of training reinforcement learning agents directly on real buildings. The proposed methodology combines synthetic data generation from established simulation frameworks, fine-tuning of time-series foundation models, and cross-platform evaluation against high-fidelity building emulators. Current progress includes a published dataset-generation study and a submitted first paper centered on zero-shot forecasting of indoor temperature and HVAC energy consumption with Tiny Time Mixers across previously unseen buildings and seasons. A later stage of the thesis will study how reinforcement learning agents can use the learned surrogates for planning and HVAC control, and whether policies trained through those models transfer back to trusted simulation environments.
Deploying reinforcement learning (RL) agents for heating, ventilation, and air-conditioning (HVAC) control across climatically diverse buildings is challenging, as policies trained in one climate often fail to generalize to others. We investigate transfer and meta-reinforcement learning for building climate control using Sinergym with a custom EnergyPlus model of a real room, comparing Double Deep Q-Network (DDQN) and Model-Agnostic Meta Learning DDQN (MAML-DDQN) across three transfer scenarios on a target very hot climate. Fine-tuned DDQN reduces temperature violations by a factor of 35 over the from-scratch baseline and improves reward by 86% over zero-shot transfer, while MAML-DDQN converges in less than half the fine-tuning episodes and exhibits substantially lower variance across seeds. Results highlight a key trade-off: MAML-DDQN is preferable at adaptation-focused deployments, while fine-tuned DDQN is superior when maximum performance is the objective.
Ulaş Filiz, Sajjad Baghaee, Mubeen Ahmed Fayyaz et al.· Signal Processing and Commun...· 0 citations
HVAC systems represent a major share of building energy consumption. Traditional control strategies are limited in coordinating energy-comfort tradeoffs across multiple zones simultaneously. Reinforcement learning (RL) offers adaptive, data-driven control that optimizes performance over time. However, deploying learned neural network controllers in safety-critical building systems remains challenging due to lack of formal safety guarantees. We propose a safety-certified deep RL framework for multi-zone residential HVAC control. Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) agents are trained in an EnergyPlus/Sinergym simulation to minimize energy consumption while maintaining thermal comfort. Post-training safety certification is performed on the PPO policy using Lipschitz-based forward invariance analysis, building on existing tools for the computation of Lipschitz constants for neural networks, to guarantee constraint satisfaction. Both agents are evaluated over an annual simulation cycle in an eight-zone variable refrigerant flow (VRF) testbed. The PPO agent achieves 67\% comfort violation reduction compared to rule-based control, while the SAC agent achieves 27.6\% energy savings. The PPO policy satisfies formal safety certification with a margin of $2.003^\circ$C. These results demonstrate the feasibility of combining reinforcement learning with post-training safety verification for multi-zone building control.
Oussama Ziadi, A. Rochd, S. I. Kaitouni et al.· 0 citations
This study adapts an open-weight reasoning model through reinforcement learning with verifiable rewards (RLVR), which converts exact offline dynamic-programming action values into dense rewards for every candidate action to adapt open-weight reasoning models to building storage scheduling.
Takumi Shioda, Kohei Terashima, T. Nagai· 1 citation· ⚡1
Multi-zone variable-air-volume control must balance thermal comfort, indoor air quality, and electricity use across several continuous actuators. Model predictive control and reinforcement learning are widely studied, but deployment typically requires building-specific modeling or training, limiting scalability. We first test whether a frontier reasoning model (an LLM trained to use additional inference-time computation) can achieve competitive VAV control from text without building-specific training. With that capability established, we then test whether TD3-guided reinforcement fine-tuning (RFT) can transfer control knowledge into a locally deployable open-weight model. Five controllers are evaluated over three summer days in a physics-based four-zone emulator. Relative to a Guideline 36-based baseline, TD3 reduced HVAC electricity by 4.5% while improving temperature and CO$_2$ compliance. Without building-specific training, GPT-5 achieved the largest reduction (6.2%) but reduced the ventilation margin. For RFT, deterministic rollouts restore a saved state, apply one candidate, and follow TD3 to score each action. Auditing a learned critic against these rollouts exposed a failure hidden by its near-perfect across-time correlation ($r=0.9998$): within-state ranking was unreliable; the critic selected the rollout-best candidate in only 5 of 10 states. Even with the rollout verifier, 200 RFT steps produced no sustained improvement in sampled-action return; the open-weight controller used more electricity than the baseline before and after training, and its five-minute predictions remained worse than persistence. GPT-5 predicted transitions far better. Exact rollout scores rank sampled actions but reveal neither next-state effects nor an improvement direction. The unchanged transition errors motivate transition-focused supervised fine-tuning before value-based RFT.
Takumi Shioda, Kohei Terashima, T. Nagai· 0 citations
Accurate state-of-charge (SoC) estimation is essential for ensuring the reliability and operation of lithium-ion batteries in electric vehicles. Traditional data-driven models failed to generalize across varying load profiles. Also, they do not handle nonlinear electrochemical behavior and long-term degradation effects of batteries. Recently, reinforcement learning–based SoC correction techniques have shown promising results in SoC predictions. But these models are based on synthetic or simplified feedback, which is unstable for real-world conditions. The goal of this work is to develop a novel approach for improving SoC prediction in real-time battery management systems. To achieve this, in this work, a reinforcement learning driven deep temporal network (RL-DTNet) is proposed for SoC prediction. The RL-DTNet consists of four major stages: (1) an adaptive feature extraction layer to construct an enriched multidomain representation; (2) a temporal attention gated recurrent unit (GRU) module to focus on the most informative timesteps under varying load disturbances; (3) a reinforcement learning self-correction module using a deep Q-network to learn how to minimize prediction errors by adjusting outputs based on real-time error feedback; and (4) a long-term degradation modeling layer to adjust SoC estimates for aging effects using a cycle-aware degradation factor. The novel contribution of this work is the integration of reinforcement learning for self-correction, temporal attention to handle dynamic dependencies, and a degradation-aware model for long-term prediction accuracy. Experimental results on real-time datasets show that RL-DTNet achieves better results of the root mean squared error, mean absolute error, and coefficient of determination (
R
2
) of 0.6377, 0.5056, and
R
2
, respectively.
S. M. Kanna, G. Narmadha, B. Sakthivel· Journal of Energy Engineerin...· 0 citations
A comfort-aware reinforcement learning (RL) framework is presented for intelligent indoor environment control in smart buildings. The system comprises three primary components: a data-driven comfort classifier, a lightweight indoor environment simulator, and a Deep Q-Network (DQN) control agent. A synthetic dataset, representing typical thermal and air-quality conditions, is pre-processed and used to train a Random Forest model that classifies occupant comfort in real time and translates these predictions into reward signals for the RL agent. The agent undergoes initial offline pre-training using a replay buffer populated with synthetic state–action–reward transitions, followed by further refinement through online interaction with the simulator to enhance sample efficiency and stability. Control performance is evaluated against a simple rule-based baseline over 100 episodes with varying internal gains and weather scenarios. The comfort classifier achieves high accuracy across all comfort categories, supporting reliable and consistent reward generation. As a result, the RL controller attains significantly higher comfort levels than the baseline while maintaining comparable or lower energy consumption. Pareto analysis indicates that the RL strategy consistently produces superior comfort–energy trade-offs, with most Pareto-optimal operating points attributed to the DQN agent. These results highlight the potential of RL as an adaptive, data-driven approach for multi-objective indoor environmental control in intelligent buildings.
Abdussalam Mohamed, Hmeda Najemeddin Musbah, Hamed H. Aly· 2026 IEEE Canadian Atlantic...· 0 citations