A reinforcement learning–based temperature control framework tailored for the evaporation process and successfully closes the gap between RL-based control and conventional PID for nonlinear high-temperature processes.
Abstract
Precise temperature regulation for the evaporation process in display manufacturing remains a significant challenge due to the nonlinearity and complexity of dynamics, safety constraints, and slow thermal responses. The conventional proportional-integral-derivative (PID) control approach has been widely used due to its simplicity; however, it often fails to maintain stability and accuracy under such nonlinear and time-varying conditions. To overcome these limitations, this paper proposes a reinforcement learning–based temperature control framework tailored for the evaporation process. A data-driven heating simulator is first developed to provide a safe and efficient virtual environment for reinforcement learning (RL) training, eliminating the need for costly and risky high-temperature experiments. A hybrid-mode RL architecture is then introduced, consisting of two agents: one for directly controlling power in the early phase and another for RL-based PID gain adjustment in the later phase. Experimental results from both the simulator and the actual heating system demonstrate that the proposed framework achieves stable temperature control. Moreover, our proposed approach significantly outperforms conventional PID control by reducing delay time by 6.90%, maximum overshoot by 94.82%, and settling time by 36.47%. For nonlinear high-temperature processes, the proposed method successfully closes the gap between RL-based control and conventional PID.
Continuous Stirred Tank Reactors (CSTRs) are widely used in chemical process industries, where precise temperature control is crucial for product quality, operational effectiveness, and safety. However, the accuracy of strict first-principle models is limited by the extremely nonlinear thermal behavior brought on by Arrhenius-based kinetics, measurement noise, transport delay, and feed fluctuations. In order to overcome this, the current study uses multivariate pilot-plant time-series data to create a high-fidelity nonlinear autoregressive model with exogenous inputs (NARX). Twin Delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor-Critic (SAC) are used in the NARX model as a reinforcement learning environment for closed-loop temperature regulation. Feed temperatures, feed flow rates, and hot-oil-bath actuation are among the various manipulated-variable combinations that are assessed. The suggested NARX–RL framework effectively tracks set points, according to the results. Both SAC and TD3 exhibit acceptable closed-loop performance, according to a comparative analysis; however, TD3 consistently outperforms SAC in tracking indices, convergence speed, and control stability across the majority of manipulated-variable combinations. Overall, the proposed NARX-based reinforcement learning technique provides a feasible option for flexible, data-driven, and practically implementable temperature regulation in lab-scale CSTR systems.
J. George, Anagha Ravikumar, Erin Joshy et al.· ACS Omega· 0 citations
The proposed NMMC-DRL strategy is evaluated through comprehensive simulations and physical experiments on a Roots-expander-based waste heat recovery platform and shows effectiveness in improving disturbance rejection, tracking accuracy, and operational robustness.
Yanjun Xiao, Ru-Qing Liu, Weiling Liu et al.· Journal of Vibration and Con...· 0 citations
Advanced control of heating and cooling systems can substantially reduce energy costs and pollution. However, real-world adoption of popular algorithms among researchers, such as model predictive control (MPC) and reinforcement learning (RL), remains limited due in part to their high deployment and commissioning costs. Here, we develop two nearly commissioning-free controllers tailored to objectives that depend linearly on the controlled thermal load, such as energy costs and pollution. The controllers require at most two thermal parameters. In representative heating simulations, controller performance is robust to large parameter specification errors, suggesting potential for deployment with no tuning. The controllers maintain good occupant comfort while achieving 43 to 98% (depending on the electricity pricing and controller variant) of the performance improvement achieved by an omniscient policy with perfect model information and forecasts. These results suggest that simple, structure-exploiting controllers may capture most of the attainable value of advanced control while avoiding the data, modeling, tuning, and computational burdens that can arise with conventional MPC or RL.
W. G. Dierking, Arash Khabbazi, L. D. Premer et al.· 0 citations
Abstract. The need to achieve sustainable production has become an urgent necessity in the conditions of stricter environmental requirements and the rise in the cost of energy worldwide. The classical proportionalintegralderivative controllers and linear Model Predictive Controllers are conventional model-based control strategies that by nature rely on precise process models and fixed optimization horizons and hence are not well suited to the dynamic, non-linear, and complex nature of the modern production environment. The current paper suggests a new adaptive control system based on Reinforcement Learning (RL), where the overall production system is optimized and controlled to achieve Sustainable Production Systems, with the multi-objective rewarding function, which aims to minimize the Overall Equipment Effectiveness (OEE), defect rate, specific energy consumption, and carbon dioxide emissions, and a physics-informed Digital Twin safety filter that stops unsafe policy execution in training and deployment. The proposed framework is evaluated on a multi-machine flexible manufacturing cell benchmark, which includes CNC milling, turning, and robotic assembly, and yields an OEE of 93.6, a defect rate of 0.9, a decrease in specific energy consumption of 27.8, and a decrease in CO 2 emissions of 23.4 compared to PID baseline controllers. Experiments with ablation prove all the above-mentioned in the necessity of each of the architectural constituents, and the Pareto frontier analysis proves that the proposed SAC agent is the best in the OEE-versus-energy trade-off space compared to all of the competing methods.
Apoorva Verma· Materials Research Proceedin...· 0 citations
Precise temperature regulation in Micro-Thermoelectric Cooler (Micro-TEC) systems is essential for applications such as polymerase chain reaction (PCR) thermal cyclers, biomedical instrumentation, and microelectronic cooling platforms. Proportional-Integral-Derivative (PID) controllers remain widely adopted for their structural simplicity, but their performance depends heavily on gain selection. Heuristic tuning has limited adaptability under nonlinear thermal dynamics, while optimization-based techniques such as genetic algorithm (GA) and particle swarm optimization (PSO) require repeated, computationally intensive searches. A consolidated PID gain knowledge repository is built by running GA and PSO offline over 2,500 Micro-TEC operating scenarios, which serves as supervised training data for an ensemble regression model composed of Random Forest, Extra Trees, and Gradient Boosting learners combined through stacking. The trained ensemble model learns the mapping between operating conditions and near-optimal PID gains, eliminating the need for repeated evolutionary optimization during deployment. On the Micro-TEC plant model, the proposed ensemble controller lowers overshoot by 37.9% (from 6.31% to 3.92%), cuts settling time by 28.0% (from 16.8 s to 12.1 s), reduces steady-state temperature error to 0.11°C (a 59.3% reduction), and selects its gains about 65 times faster (15 ms against 980 ms) than a conventional PSO-tuned PID. The results show that the framework bridges classical PID control and data-driven learning, providing a scalable, computationally efficient approach for scenario-aware gain scheduling in nonlinear thermal systems.
M. Anitha, S. B. Prusty· Journal of Artificial Intell...· 0 citations
Laser powder bed fusion (LPBF) is a metal additive manufacturing process where temperature stabilization is of vital importance to avoid defects such as distortion and cracking. Existing control methods require manual tuning, increasing the risk of part failure when printing complex geometries. This paper introduces a dual-loop, data-driven control strategy to stabilize the surface temperature, ensuring robustness and near-optimal performance in the presence of disturbances. The proposed method integrates (i) an in-layer linear output feedback control with gains optimized through policy gradient, and (ii) a layer-to-layer feedforward control combining temperature trajectory optimization and iterative learning control. Simulation results show that the multi-scale controller effectively stabilizes the temperature even under significant model mismatch and measurement noise. Experimental results demonstrate that a simplified, hardware-constrained version of this method matches the state-of-the-art performance of in-situ data-driven methods, reducing mean tracking error by 3.4% and mean input-constraint violation by 47.5% relative to a Bayesian Optimization-tuned baseline. For this physical LPBF validation, the controller is tuned entirely offline using uncontrolled print data from a single calibration layer. Our experiments also demonstrate a new class of high-frequency excitation dynamics that result in reduced vector head swelling, opening up new avenues of research in the additive manufacturing community. This work marks one of the first successful applications of sim-to-real policy optimization in LPBF processes.
Jun-Feng Lin, Riccardo Zuliani, Barış Kavas et al.· 0 citations