PTRL: Prior Transfer Deep Reinforcement Learning for Legged Robots Locomotion

Abstract

Although reinforcement learning (RL) has shown great potential in legged robot motion control, traditional methods face two major bottlenecks: the strategy training of a single robot requires a large amount of computational resources, and the trained models are highly specific and complex for direct transfer and application. To address this key issue, this study proposes the Prior Transfer Reinforcement Learning (PTRL) framework, which improves training efficiency and model generalization ability through cross-robot knowledge transfer. The core contribution of this study is the construction of a three-stage learning paradigm of ‘pre-training, transfer, fine-tuning.’ Specifically, we first train the source robot strategy based on the Proximal Policy Optimization (PPO) algorithm and then achieve efficient knowledge transfer to the target robot by selectively freezing the key layers of the policy executor network. Through systematically designed multi-robot platform comparative experiments, it was verified that this method can reduce the training time, and the quantitative relationship between the proportion of frozen network layers and the transfer effect was revealed. The experimental results show that the proposed method significantly improves the performance of legged robots in walking tasks, demonstrating its strong applicability and advantages.