Optimizing 3D UAV navigation via a high-efficiency hybrid RRT*-DQN approach

Abstract

Path-planning for autonomous unmanned aerial vehicles (UAVs) must be highly efficient, safe, and dynamically feasible for manoeuvring through 3D environments densely packed with obstacles. Traditional heuristic and sampling methods for path-planning often struggle to ensure spatial safety while simultaneously optimizing path quality. They often produce rough and inefficient paths, which lead to instability problems when the vehicle is airborne and higher battery consumption due to longer flight times. We tackle the challenge with this paper by developing a hybrid path-planning method, rapidly-exploring random tree star (RRT*)-deep Q-network (DQN), which combines the fast global search capability of RRT* with the deep learning-based heuristic prediction of a DQN. To demonstrate the merits of our approach, the new system has been tested in four very different 3D space scenarios and its performance compared to seven other leading methods (RRT*, informed RRT*, potential field-based RRT*, heuristic potential optimization RRT*, adaptive grey wolf optimizer, A*, and improved Dijkstra), each method being run 30 times independently with random seeds to allow for a meaningful statistical analysis. The numerical data clearly show that RRT*-DQN outperforms other methods in terms of path optimality almost all the time, with path efficiency reaching up to 97.7% and a trajectory smoothness exceeding 99.86%, while remaining only slightly less competitive in terms of obstacle clearance. Deep reinforcement learning-based inference does bring about a modest computational overhead during the first offline planning stage; however, this is a modest computational overhead, given that it directly produces paths suitable for subsequent flight execution that avoid the kinematically challenging manoeuvres typical of traditional planners. Thus, the RRT*-DQN algorithm offers a very strong candidate for UAVs operating in real-world environments by trading increased offline computational time for improved path quality and potentially more efficient physical execution.