Aug 2026· Italian National Conference on Sensors· Vol 26, pp. 5031· 0 citations· 37 references
Medicine
TL;DR
This paper proposes an edge-cloud collaborative physics-informed reinforcement learning framework for production data center HVAC control that integrates a physics-informed cold-start solution using Adaptive Particle Swarm Optimization, a three-time-scale edge–cloud architecture, and a constraint-aware safe projection layer that embeds thermal safety hard constraints directly into the neural network policy.
Abstract
The sustained growth of cloud computing and AI training workloads drives data center expansion. Optimizing their control is therefore critical for reducing operational costs. Edge real-time control is indispensable for guaranteeing thermal safety, data sovereignty, and offline availability. Yet deploying Deep Reinforcement Learning (DRL) in production Heating, Ventilation, and Air Conditioning (HVAC) environments confronts cold-start risks, edge–cloud computational asymmetry, and multi-objective conflicts spanning energy efficiency, electricity cost, and thermal safety. To address these challenges, this paper proposes an edge-cloud collaborative physics-informed reinforcement learning framework for production data center HVAC control. The framework integrates a physics-informed cold-start solution using Adaptive Particle Swarm Optimization (APSO) to generate physically constrained initial policies on a gray-box digital twin without expert demonstration data, a three-time-scale edge–cloud architecture coordinating minute-level edge Soft Actor-Critic (SAC) real-time inference, weekly edge APSO online model identification, daily cloud Non-dominated Sorting Genetic Algorithm III (NSGA-III) thermal storage scheduling, and a constraint-aware safe projection layer that embeds thermal safety hard constraints directly into the neural network policy. The framework is validated through a seven-month production deployment spanning the complete summer-to-winter transition, comprising approximately 3.2 million sensor records and evaluated with rigorous statistical methods.
Driven by artificial intelligence and cloud computing, hyperscale data centers are becoming one of the fastest-growing electrical loads worldwide and are increasingly recognized as a new class of flexible loads capable of supporting demand response (DR) and the integration of variable renewable energy (VRE). However, their two principal control levers—IT workload scheduling and cooling system operation—have traditionally been managed in a decoupled manner, leaving both energy efficiency and demand-side flexibility under-exploited. This paper proposes a deep reinforcement learning (DRL) framework that jointly co-schedules computing and thermal resources so that a hyperscale data center can operate as a grid-interactive flexible load. We formulate the joint problem as a constrained Markov Decision Process and develop an actor-critic algorithm combining Deep Deterministic Policy Gradient with a safety shield mechanism to guarantee thermal constraint satisfaction during both training and deployment. A high-fidelity digital twin simulation environment enables safe Sim-to-Real training. Extensive experiments demonstrate that the proposed approach reduces total electricity consumption by 18-25% compared to baseline controllers, cuts thermal violations by over 90%, and maintains service level agreement compliance, while broadening the controllable power envelope of the facility to provide a technical basis for participating in DR programs and aligning data-center power profiles with renewable generation. The framework bridges IT-side and facility-side control and supports the evolution of hyperscale data centers from passive electricity consumers toward active, grid-interactive participants in renewable-penetrated power systems.
The proliferation of distributed energy resources at the edge of distribution networks provides substantial flexibility for virtual power plant (VPP) operation. However, existing methods often rely on aggregate load information and homogeneous scheduling policies. They, therefore, overlook device-specific response characteristics, heterogeneous response times, and operational safety constraints. This paper presents EDGE-VPP, an end-to-end scheduling framework that connects fine-grained load perception with safety-aware decision-making across multiple temporal scales. At the perception layer, a Load Decomposition Transformer (LDT) uses learnable multi-frequency positional encodings and device-specific attention heads. It jointly detects appliance states and disaggregates device power from aggregate measurements. At the coordination layer, a three-tier cloud–edge–device architecture assigns sub-second emergency response to devices, minute-level economic dispatch to edge controllers, and hour-ahead planning to the cloud. Bidirectional information exchange mitigates conflicts among these control layers. At the optimization layer, multi-constraint proximal policy optimization factorizes continuous and discrete actions. Adaptive Lagrange multipliers enforce voltage and current limits, while two value estimators stabilize policy learning. Experiments on REDD, UK-DALE, and a self-constructed VPP dataset show that LDT reduces mean absolute error by up to 6.86% and improves the F1-score by 3.51% over the Transformer baseline. The complete EDGE-VPP framework also achieves the lowest operating cost and the fewest constraint violations among the evaluated scheduling methods.
Experimental results demonstrate that DL-EATS achieves the lowest energy consumption, shortest makespan, minimal SLA violation rate, and highest resource utilization, representing an 18.5% improvement in energy efficiency over the next best method and substantial gains across all performance metrics.
Abdulmumini Adamu, A. A. Abdulwasiu· Journal of Science Research...· 0 citations
Green low-carbon data centers operate as coupled cyber-energy systems whose dispatch must coordinate renewable generation, grid exchange, battery storage, cooling load, flexible computing workload, carbon-intensity signals, and reliability constraints. This study develops and evaluates a safety-constrained deep reinforcement learning framework for source–load–storage coordinated operation of a grid-connected green data center. The operating problem is formulated as a constrained Markov decision process with state variables describing the IT load, deferrable workload backlog, renewable availability, electricity price, marginal carbon intensity, battery state of charge, server-room temperature, reserve margin, and calendar context. The action space covers grid import and export, renewable utilization, storage charge and discharge, workload shifting, and cooling control. The learning architecture combines a constrained actor–critic policy, adaptive Lagrangian safety critics, and a control barrier function (CBF)-based action shield that projects unsafe actions onto an explicitly defined operating set before plant execution. The shield is specified as a low-dimensional quadratic projection over state-dependent SOC, thermal, reserve, SLA, and grid-interface constraints, while cumulative risks are priced through Lagrangian safety budgets during policy training. The evaluation uses a controlled and auditable benchmark simulation with normalized public-data-compatible profiles, declared scenarios, random seeds, neural-network settings, and mechanism-matched baselines; it is not a telemetry-based verification or hardware certification of a deployed data center. Within this declared benchmark, the proposed safe DRL controller produces a simulated 13.1% emission reduction relative to the Rule-based controller, 95.8% renewable utilization, a normalized annual cost of 0.91, and fewer boundary contacts than the tested unconstrained, Lagrangian-only, and shield-only PPO variants. These percentages are simulator outputs relative to the stated benchmark and must not be interpreted as measured field savings. The results show how separating reward learning, cumulative safety pricing, and one-step engineering projection changes low-carbon dispatch within the specified model.
Zheng Shi, Min Xu, Ziyu Fu et al.· Energies· 0 citations
Sustainable power systems increasingly require scheduling methods that can coordinate renewable generation, distributed flexibility, and conventional energy-conversion units under forecast uncertainty while directly supporting carbon-emission reduction. However, many data-driven scheduling models still enforce operational constraints through soft penalties or post-processing corrections, which may lead to infeasible schedules during deployment and weaken their reliability in digital-shadow-assisted operation. In addition, conventional cost-oriented scheduling objectives do not explicitly account for the carbon impact of real-time imbalances caused by forecast errors. To address these challenges, this paper proposes a digital-shadowing-enabled deep learning framework for carbon-aware day-ahead scheduling of integrated energy systems. The main methodological contribution is a feasibility-by-design neural decoder that embeds hard physical constraints directly into the network forward pass. By classifying devices into non-memory fast units, non-memory ramp-limited units, and memory-type storage devices, the decoder applies tailored transformations to enforce capacity limits, ramp-rate restrictions, state-of-charge dynamics, and terminal energy consistency by construction. Therefore, the generated schedules are physically feasible without relying on post-hoc repair. In parallel, a carbon-first objective is developed to minimize both scheduled emissions and imbalance-driven emissions, allowing the scheduler to reduce not only planned carbon output but also the carbon impact of real-time corrective actions. Forecast uncertainty is represented through a digital shadow that stores historical forecast-error patterns and generates augmented training scenarios. Case studies based on U.K. data show that the proposed framework produces fully feasible schedules and reduces annual CO2 emissions by approximately 4.0% compared with a forecast-driven baseline, with larger benefits during high-demand periods. These results demonstrate that combining digital shadowing, constraint-embedded neural decoding, and carbon-aware optimization provides a practical and reliable pathway for low-carbon smart-grid scheduling under uncertainty.
Yinuo Yang, Minglei You, Marco Rivera et al.· Technologies· 0 citations
HVAC systems represent a major share of building energy consumption. Traditional control strategies are limited in coordinating energy-comfort tradeoffs across multiple zones simultaneously. Reinforcement learning (RL) offers adaptive, data-driven control that optimizes performance over time. However, deploying learned neural network controllers in safety-critical building systems remains challenging due to lack of formal safety guarantees. We propose a safety-certified deep RL framework for multi-zone residential HVAC control. Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) agents are trained in an EnergyPlus/Sinergym simulation to minimize energy consumption while maintaining thermal comfort. Post-training safety certification is performed on the PPO policy using Lipschitz-based forward invariance analysis, building on existing tools for the computation of Lipschitz constants for neural networks, to guarantee constraint satisfaction. Both agents are evaluated over an annual simulation cycle in an eight-zone variable refrigerant flow (VRF) testbed. The PPO agent achieves 67\% comfort violation reduction compared to rule-based control, while the SAC agent achieves 27.6\% energy savings. The PPO policy satisfies formal safety certification with a margin of $2.003^\circ$C. These results demonstrate the feasibility of combining reinforcement learning with post-training safety verification for multi-zone building control.
Oussama Ziadi, A. Rochd, S. I. Kaitouni et al.· 0 citations