Improved Steady-State Error Tracking for Rotary Inverted Pendulum Using Integral-Augmented Deep Deterministic Policy Gradients (I-DDPG)

Abstract

The stabilization and positional tracking of highly underactuated, nonlinear systems, such as the Rotary Inverted Pendulum, present significant challenges particularly regarding steady-state drift caused by static friction. Although classical control approaches can achieve equilibrium, they are highly susceptible to unmodeled physical disturbances, resulting in a permanent offset of the rotary arm. Standard Deep Reinforcement Learning techniques also struggle to eliminate this drift, where the pendulum is upright but the arm remains off-center. This work proposes a novel control architecture named Integral-Augmented Deep Deterministic Policy Gradient (I-DDPG) which combines classical integral control theory with deep neural networks to explicitly augment the agent’s observation space with the time-integral of the tracking error. The Quanser QUBE-Servo 3 platform is used as a benchmark for the proposed framework and is evaluated against standard DDPG, TD3, LQR, and state-feedback controllers. The real-time hardware deployment and rigorous disturbance simulations demonstrate that the I-DDPG agent successfully breaks the static friction deadlock, reliably converging to minimum steady-state error. The proposed approach significantly outperforms both classical and baseline RL controllers, providing a highly robust approach.