Skip to content
Conference

Research on local path planning for UUV based on improved Q-learning algorithm

Oct 2026 · International Conference on Computer Vision, Robotics and Automation Engineering · Vol 14353, pp. 143531Q - 143531Q-8 · 0 citations · 10 references
Engineering

TL;DR

Compared to the traditional Q-Learning algorithm, the improved algorithm significantly reduces the number of episodes required to find the optimal path, produces smoother paths closer to the optimal solution, and achieves substantial improvements in convergence speed, exploration efficiency, and path planning quality.

Abstract

To address the shortcomings of the traditional Q-Learning algorithm in UUV local path planning, such as slow convergence, low exploration efficiency, and redundant paths, this paper proposes an improved Q-Learning algorithm. The algorithm optimizes the Q-value table update rule by incorporating a multi-step Temporal Difference (TD) update method. It constructs an 8-direction exploration mechanism to complete the action space and optimizes the exploration strategy. Furthermore, it integrates the A* algorithm to design a heuristic dynamic learning rate adjustment strategy, enhancing algorithm performance from three aspects. MATLAB simulation comparison experiments were conducted in a 20×20 complex grid environment. The results demonstrate that compared to the traditional Q-Learning algorithm, the improved algorithm significantly reduces the number of episodes required to find the optimal path, produces smoother paths closer to the optimal solution, and achieves substantial improvements in convergence speed, exploration efficiency, and path planning quality. It can be effectively applied to local path planning tasks for UUVs in unknown underwater environments.

View source

Similar papers

Oct 2026

UAV path planning via fusion of enhanced RRT* and path-guided Q-learning

This study proposes an unmanned aerial vehicle path-planning framework that integrates improved Rapidly-exploring Random Tree Star (improved RRT*) and Q-learning algorithms (Improved Rapidly-exploring Random Tree Star (IRRT*)-IQL) to address challenges such as low path-planning efficiency and susceptibility to local op...

Shi-Kai Shao, Xing-Wang Li, Hui-Peng Zhang et al. · 0 citations
Open access Sep 2026

Research on Robot Path Planning Based on Improved A* Algorithm

Robot path planning is a key core technology for realizing autonomous navigation of robots. Among them, the A* algorithm has been widely used due to its good balance between path optimality and search efficiency. However, the traditional A* algorithm faces problems such as excessive expansion nodes, numerous path infle...

Shao-Wei Hao · 2 citations

Performance evaluation and optimization of a mobile robot path-planning model based on reinforcement learning

Autonomous navigation in mobile robotics faces significant challenges in dynamic, unstructured environments, as re-planning strategies often lack time efficiency when recomputing around new obstacles. Traditional path-planning algorithms typically rely on static, pre-mapped environments and rigid geometric rules, which...

Luis Angel Castañeda Barron, Miguel Andres Meillon Ramirez, Cristóbal Armiñan Ibarra et al. · 0 citations
Conference Open access Sep 2026

Research on UAV path planning based on an improved A* algorithm

UAV path planning is a crucial component of autonomous navigation and logistics delivery, and its performance directly affects both operational safety and efficiency. However, the standard A* algorithm often suffers from redundant node expansions and limited path smoothness in cluttered environments, which constrains r...

Duo Ding, Da-Long Liu · 0 citations
Open access 2026

N-DQN-RRT: A Hybrid Deep Q-Network Guided RRT Algorithm for an Efficient Path Planning in Two-Dimensional Environments

Autonomous navigation requires path-planning algorithms that can find safe and efficient routes while avoiding obstacles. Classical sampling-based planners such as Rapidly-exploring Random Tree (RRT) and RRT* are widely used because they are flexible and easy to apply in different environments. However, these methods o...

Wai Yan Naing Win, K. Kanagalakshmi, K. Priyadharshini et al. · 0 citations
Open access Sep 2026

A hybrid-enhanced walrus optimizer for mobile robot path planning

Path planning is essential to autonomous navigation in mobile robots, and its performance directly affects task execution efficiency and operational safety. To address the limited convergence accuracy, susceptibility to local optima, and limited late-stage exploitation capability of the walrus optimizer (WO) in path pl...

Min Wu, Bing-Xing Liu, Kun-Ting Zhao et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.