Evolution of algorithms for managing complex systems
TL;DR
A comparative analysis of the evolution of control methods for complex systems: from classical PID controllers to modern approaches based on Reinforcement Learning and Deep Reinforcement Learning, showing that RL/DRL systems surpass PID in energy efficiency and adaptability in dynamic environments, but require significant computational resources and pose verifiability problems.
Abstract
The article presents a comparative analysis of the evolution of control methods for complex systems: from classical PID controllers to modern approaches based on Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL). This study aims to pinpoint the strengths and weaknesses of various control algorithm classes and validate where they can be effectively applied to practical tasks such as autonomous driving, industrial automation, energy, and robotics. The methodology includes a systematic review of 15 scientific sources, structured comparison based on criteria of accuracy (steady-state control error), stability (disturbances and parameter changes), and adaptability (self-tuning capability), analysis of numerical data from practical implementations. The scientific novelty lies in the development of a comprehensive methodology for comparative evaluation of classical and intelligent control systems with quantitative justification of application areas: PID provides accuracy of ±1–3% for linear objects, RL ±0.5–2% with adaptation in 10–50 iterations, DRL ±0.1–0.5% when working with high-dimensional data. The main results show that RL/DRL systems surpass PID in energy efficiency by 10–40% (confirmed by the example of Google data center cooling management with 40% savings) and adaptability in dynamic environments, but require significant computational resources (days or weeks of GPU training) and pose verifiability problems. The practical significance lies in providing control system designers with evidence-based recommendations for choosing the type of control system, considering the complexity of the controlled object, adaptability requirements, and available computational resources.