Reinforcement Learning for Real-Time Auto-Tuning of Robot’s Controllers

Abstract

This research introduces an innovative control technique for Series Elastic Actuators (SEAs) that utilizes Reinforcement Learning (RL) to address the shortcomings of previously fixed-gain adaptive controllers, which are a hybrid of State Feedback Control (SFC) and Model Reference Adaptive Control (MRAC) by using Lyapunov Stability Analysis. This controller is optimized by adjusting the adaptation factor. b. This study presents an intelligent agent based on reinforcement learning to find the value of b with a dynamic auto-tuner. It trains via the Soft Actor-Critic (SAC) algorithm for 100,000 time steps. A comparison between the two methods was presented according to simulation results under different conditions; the RL-based controller shows much better tracking accuracy, how quickly it reaches the target output, and how little control torque it uses, where the agent's policy could automatically adjust in real-time based on system conditions, such as uncertainties and disturbances, where it has a settling time of 1.7 seconds, while the fixed parameter controller has a 1.95-second settling time, resulting in a reduction of 15.3%. It also lowers the control torque caused by disturbances by 19.5% compared to the fixed parameter controller, which has a control torque of 3.99 Nm, while the maximum control torque for the RL-optimized controller is 3.21 Nm.