Optimizing 3D UAV Navigation via a High Efficiency Hybrid RRT*-DQN Approach
Path-planning for Autonomous Unmanned Aerial Vehicles (UAVs) must be not only highly efficient and safe but also dynamically feasible to manoeuvre through 3D environments densely packed with obstacles. Traditional heuristic and sampling methods for path-planning are known to have a hard time ensuring spatial safety while still optimising the path. They often end up with quite rough, inefficient paths, which lead to instability problems when the vehicle is airborne and higher battery consumption due to longer flight times. We tackle the challenge with this paper by developing a hybrid path-planning method, RRT*-DQN, which combines the fast global search capability of Rapidly-exploring Random Tree Star (RRT*) with the deep learning-based heuristic prediction of a Deep Q-Network (DQN). To demonstrate the merits of our approach, the new system has been tested in three very different 3D space scenarios and its performance compared to seven other leading methods (RRT*, Informed RRT*, PF-RRT*, HPO-RRT*, AGWO, A*, and Improved Dijkstra), each method being run 30 times independently with random seeds to allow for a meaningful statistical analysis. The numerical data clearly show that RRT*-DQN outperforms other methods in terms of path optimality almost all the time, with path efficiency reaching up to 97.8\% and a never-before-seen trajectory smoothness of over 99.6\%, while remaining only slightly less competitive in the obstacle-clearance dimension of the problem. Deep reinforcement learning-based inference does bring about a slight increase in computational time during the first offline planning stage; however, this is quite a small price to pay, given that it directly produces flight-ready paths that avoid the kinematically challenging manoeuvres typical of traditional planners. Thus, the RRT*-DQN algorithm offers a very strong candidate for UAVs operating in real circumstances.