Stabilizing Dynamical Systems with Model-Free Control: A Deep Q-Network Approach
TL;DR
This approach demonstrates the potential of DQN in scenarios where system assumptions and prior knowledge are impractical or unavailable, while acknowledging that the empirical success in the benchmark environment does not constitute formal control-theoretic stability guarantees.
Abstract
Modeling dynamical systems is a challenging task, especially in the presence of limited sensor feedback and external disturbances, which can render control systems unstable. Traditional control methods often require detailed mathematical models and assumptions about the system's behavior, making them unsuitable for complex, nonlinear systems or those with limited sensor feedback. Model-free control offers a promising alternative by eliminating the need for explicit system modeling. This paper explores the application of a Deep Q-Network (DQN) algorithm to stabilize an inverted pendulum using raw pixel data as the sole state feedback in a discrete-action setting. The study demonstrates the effectiveness of reinforcement learning in addressing control problems in environments where state variables cannot be directly derived due to sensor limitations or failures. By leveraging a model-free formulation, this approach demonstrates the potential of DQN in scenarios where system assumptions and prior knowledge are impractical or unavailable, while acknowledging that the empirical success in the benchmark environment does not constitute formal control-theoretic stability guarantees.