Jul 2026· 2026 IEEE Canadian Atlantic Ocean Symposium (CAOS)· pp. 185-188· 0 citations· 15 references
Abstract
A comfort-aware reinforcement learning (RL) framework is presented for intelligent indoor environment control in smart buildings. The system comprises three primary components: a data-driven comfort classifier, a lightweight indoor environment simulator, and a Deep Q-Network (DQN) control agent. A synthetic dataset, representing typical thermal and air-quality conditions, is pre-processed and used to train a Random Forest model that classifies occupant comfort in real time and translates these predictions into reward signals for the RL agent. The agent undergoes initial offline pre-training using a replay buffer populated with synthetic state–action–reward transitions, followed by further refinement through online interaction with the simulator to enhance sample efficiency and stability. Control performance is evaluated against a simple rule-based baseline over 100 episodes with varying internal gains and weather scenarios. The comfort classifier achieves high accuracy across all comfort categories, supporting reliable and consistent reward generation. As a result, the RL controller attains significantly higher comfort levels than the baseline while maintaining comparable or lower energy consumption. Pareto analysis indicates that the RL strategy consistently produces superior comfort–energy trade-offs, with most Pareto-optimal operating points attributed to the DQN agent. These results highlight the potential of RL as an adaptive, data-driven approach for multi-objective indoor environmental control in intelligent buildings.
No single AI weather model excels at all variables, pressure levels, and lead times. Rather than building yet another architecture, we reframe the forecasting problem as one of coordination. Here we present Feitian Adaptive Ensemble Weather (FTAE-Weather), a lightweight framework that learns, through deep reinforcement learning, when and where to trust each member of an open pool of pretrained forecasters. A tactical Weight-Agent reads the current atmospheric state and assigns variable- and horizon-specific fusion weights, while a strategic Evolve-Agent periodically prunes underperforming models and absorbs newly released ones. Asynchronous prediction caching keeps training cost independent of the slowest constituent model. Adding fewer than 0.01 percent extra parameters, FTAE-Weather reduces RMSE by from 17.2 percent to 78.3 percent over the best individual model in 10 atmospheric variables and outperforms conventional ensemble baselines across lead times from 72 to 360 hours. The framework thus converts a growing, fragmented inventory of specialist models into a single prediction system that strengthens as the field of AI weather forecasting releases new architectures-turning model diversity from a coordination challenge into a compounding scientific advantage.
HVAC systems represent a major share of building energy consumption. Traditional control strategies are limited in coordinating energy-comfort tradeoffs across multiple zones simultaneously. Reinforcement learning (RL) offers adaptive, data-driven control that optimizes performance over time. However, deploying learned neural network controllers in safety-critical building systems remains challenging due to lack of formal safety guarantees. We propose a safety-certified deep RL framework for multi-zone residential HVAC control. Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) agents are trained in an EnergyPlus/Sinergym simulation to minimize energy consumption while maintaining thermal comfort. Post-training safety certification is performed on the PPO policy using Lipschitz-based forward invariance analysis, building on existing tools for the computation of Lipschitz constants for neural networks, to guarantee constraint satisfaction. Both agents are evaluated over an annual simulation cycle in an eight-zone variable refrigerant flow (VRF) testbed. The PPO agent achieves 67\% comfort violation reduction compared to rule-based control, while the SAC agent achieves 27.6\% energy savings. The PPO policy satisfies formal safety certification with a margin of $2.003^\circ$C. These results demonstrate the feasibility of combining reinforcement learning with post-training safety verification for multi-zone building control.
Oussama Ziadi, A. Rochd, S. I. Kaitouni et al.· 0 citations
Deploying reinforcement learning (RL) agents for heating, ventilation, and air-conditioning (HVAC) control across climatically diverse buildings is challenging, as policies trained in one climate often fail to generalize to others. We investigate transfer and meta-reinforcement learning for building climate control using Sinergym with a custom EnergyPlus model of a real room, comparing Double Deep Q-Network (DDQN) and Model-Agnostic Meta Learning DDQN (MAML-DDQN) across three transfer scenarios on a target very hot climate. Fine-tuned DDQN reduces temperature violations by a factor of 35 over the from-scratch baseline and improves reward by 86% over zero-shot transfer, while MAML-DDQN converges in less than half the fine-tuning episodes and exhibits substantially lower variance across seeds. Results highlight a key trade-off: MAML-DDQN is preferable at adaptation-focused deployments, while fine-tuned DDQN is superior when maximum performance is the objective.
Ulaş Filiz, Sajjad Baghaee, Mubeen Ahmed Fayyaz et al.· Signal Processing and Commun...· 0 citations
A Large Language Model-enhanced Autonomous Reinforcement Learning Penetration Testing framework that leverages the domain knowledge embedded in a Large Language Model to perform tactical planning, thereby pruning the original action space into a compact set of candidate actions.
Rising electricity demand and high energy costs, particularly in rapidly growing regions, highlight the need for more efficient building energy management. Air conditioning (AC) systems account for a substantial portion of electricity consumption, yet conventional fixed-temperature control lacks adaptability to varying environmental conditions. This study introduces a Smart Air Conditioning Management System based on a Deep Q-Network (DQN) agent capable of dynamically balancing energy use and thermal comfort. The system integrates IoT-enabled environmental sensing and actuation, using ESP32 and Raspberry Pi microcontrollers, along with user feedback for automated control of AC units. Initial training and evaluation were conducted in recreated classroom-laboratory environments using simulations in EnergyPlus, and experimental deployment validated performance in real rooms. Simulation results show up to 81% energy reduction under low comfort prioritization, while field deployment achieved approximately 60% savings. These findings demonstrate that reinforcement learning enables adaptive AC control, offering a scalable approach to energy-efficient building management.
Jason Harvey Lorenzo, Justin Kyle O. Ricafort, E. Q. Macabebe· IOP Conference Series: Earth...· 0 citations
This paper proposes H-UPF (Hybrid Universal Policy with Forecasting), a hybrid intelligent framework for scalable sequential decision-making in heterogeneous environments under uncertainty. The architecture integrates probabilistic multi-horizon forecasting via a Temporal Fusion Transformer with continuous control via Proximal Policy Optimization, embedding predictive quantile distributions directly into the agent’s state representation. A Dynamic Adaptation Layer normalizes observations relative to instance-specific scales, enabling zero-shot policy transfer across environments with 18.5× variability in operating characteristics — without inter-agent communication or per-instance retraining. Validated on two real-world residential energy management datasets (REFIT: 20 UK households; CityLearn: 6 US buildings with real PV profiles), the framework achieves 88.4% of the theoretical optimum in zero-shot transfer, outperforming meta-learning (MAML-PPO) by 8.4 percentage points (Wilcoxon p = 0.003, Cohen’s d = 1.42). Ablation analysis identifies the adaptation layer as the dominant contributor (−16.2 p.p. upon removal), while probabilistic forecasting adds +6.8 p.p. through proactive scheduling. The learned policy is robust to reward parameter variations (≤3.2 p.p. sensitivity across 5× range) and supports practical deployment: 9.8 h one-time training, 18.4 ms inference per control step.
A. Tokhmetov, L. Tanchenko, M. Kenesbai· Bulletin of Manash Kozybayev...· 0 citations