Proactive UAV Routing with Optical Flow and Reinforcement Learning for Adaptive Surveillance

Abstract

The rapid advancement of AI and sensor technologies has brought unprecedented capabilities while introducing new vulnerabilities, particularly in critical resources and defense systems, causing significant challenges to national security. Adversaries may exploit these technologies to undermine surveillance systems, disrupt defense operations, and take advantage of vulnerable areas. Addressing these challenges requires comprehensive, data-driven modeling approaches that enable proactive surveillance and robust defense strategies. This study presents an optical flow (OF)-based routing policy for unmanned aerial vehicles (UAVs), enabled by Markov Decision Processes (MDPs) and leveraging real-time sensor data within a game-theoretic framework. By integrating OF values—a measure of dynamic motion patterns in the environment—into reinforcement learning framed by MDPs, the proposed approach anticipates threats and enhances adaptability to rapidly changing conditions while accounting for adversarial behaviors. This methodology enables UAVs to respond intelligently to dynamic scenarios, such as shifting threats or unforeseen intruders, while maintaining comprehensive and effective surveillance coverage. The framework accounts for various adversarial behaviors, including dive bombers (high-speed, targeted threats), random movers (unpredictable, erratic patterns), and strategic adversaries (deliberate, calculated interference). Validation is conducted using a high-fidelity physics-based simulation designed to emulate realistic operational scenarios. These scenarios incorporate diverse environmental conditions and adversarial interference, ensuring comprehensive testing of the methodology. The findings demonstrate the potential of integrating optical flow analysis with reinforcement learning, enhancing situational awareness, and improving the effectiveness of surveillance operations against diverse and evolving threats.