A hierarchical reinforcement learning approach for robot navigation integrating LiDAR priors and the options framework

Abstract

Autonomous navigation of robots in complex indoor environments faces challenges such as long-term decision-making, sparse rewards, and low sample efficiency. Traditional end-to-end deep reinforcement learning methods often suffer from poor policy convergence and weak generalization ability when handling long-term tasks. This paper proposes a hierarchical reinforcement learning method based on an option framework. This method formalizes the navigation task as a semiMarkov decision process and constructs a two-layer decision architecture with collaboration between a high-level manager and a low-level worker. The core innovations are: proposing an option definition method based on semantic priors of the indoor environment; designing a hybrid reward mechanism that integrates sparse extrinsic rewards and dense intrinsic rewards; and adopting a three-stage course learning training process of “lower-level pre-training—higher-level training— joint fine-tuning”. Experimental results show that the proposed method achieves a task success rate of 67.0%, improves the average reward by approximately 45.2%, and reduces the number of interaction steps required for convergence by approximately 56.3%.