Perturbation-Resilient Autonomous Navigation with Distributionally Robust Reinforcement Learning

A Distributionally Robust Multi-agent Reinforcement Learning Framework for Intelligent Intersection Control

Multi-agent reinforcement learning (MARL) has emerged as a promising approach for traffic signal control. However, standard MARL policies typically optimize for expected returns under nominal conditions, leaving them highly vulnerable to spatial-temporal demand shifts and catastrophic congestion under adverse scenarios. To address this critical limitation, this paper proposes an algorithm-agnostic Distributionally Robust (DR) MARL framework integrating an adaptive Contextual-Bandit Worst-Case Estimator (CB-WCE). Operating on a slower timescale, the CB-WCE co-evolves with the traffic controllers by dynamically generating adversarial demand mixtures during training. This steers the learning process to fortify policies against bottleneck scenarios without requiring modifications to the underlying MARL architectures. The framework is evaluated across value-based, actor-critic, and policy-gradient methods on both a synthetic 5x5 grid and a heterogeneous Monaco City network. Empirical results demonstrate that the DR framework prevents unbounded queue growth and profoundly enhances both worst-case robustness and average-case efficiency. Notably, for the Proximal Policy Optimization (PPO) architecture in the Monaco environment, on average, robust retraining reduced the worst-case queue length by 74.39% and improved the average-case network-wide queue length by 75.45%. Furthermore, the retrained policies exhibit strong zero-shot generalization to unseen traffic distributions, highlighting the framework's scalability and potential for resilient real-world urban deployment.

Disturbance-Aware Hybrid Learning for Robust and Adaptive UAV Flight in Extreme Winds

Sample-Efficient Autonomous Surface Vehicle Navigation with Enhanced Experience Replay

This paper proposes a sample-efficient reinforcement learning framework for navigation of Autonomous Surface Vehicles (ASVs) in complex, obstacle-dense environments. Learning reliable navigation policies in such settings is challenging due to unsafe early exploration and high sample complexity. To address these issues, we integrate Distributional Reinforcement Learning with Deep Q-learning from Demonstrations (DQfD) and Count-Based Experience Replay (CbER), enabling effective learning from both expert priors and prioritized recent experiences. Furthermore, we critically evaluate the integration of Hindsight Experience Replay (HER) in navigational setups that already employ dense shaping rewards. Extensive simulations demonstrate that while HER offers limited marginal utility in the presence of strong shaping signals, the combined framework accelerates convergence by up to 6.3× compared to baselines.

Flow-aware Optimal Navigation in Unsteady Flows through Reinforcement Learning

A reinforcement learning approach using the TD3 algorithm to train autonomous agents to reach arbitrary targets within a parametric, chaotic double-gyre flow is presented and a trade-off in sensor utility is revealed: velocity-aware agents optimize energy efficiency, whereas vorticity sensors provide superior structural mapping and achieve better target proximity.

Waypoint Navigation of a 2D Drone in Stochastic Environment via Regularised Proximal Policy Optimisation

Autonomous navigation of aerial vehicles under stochastic wind disturbances is a challenging control problem that classical fixed‐gain methods struggle to handle reliably. We present a three‐stage training pipeline that combines imitation learning and reinforcement learning for 2D drone waypoint navigation: a cascaded PD expert collects quality‐filtered demonstration trajectories; behaviour cloning (BC) warm‐starts the policy by supervised imitation of those demonstrations; and Proximal Policy Optimization (PPO) fine‐tunes it through environment interaction, with an auxiliary BC loss retained at every update to prevent the RL gradient from erasing the imitation prior. Trained with randomly sampled targets, the policy generalises to unseen waypoints under continuously drifting wind, maintaining stable, near‐level flight. A systematic ablation across three random seeds shows that the strength of the auxiliary BC loss is the most critical design choice: removing it yields a policy that maximises shaped reward yet fails to navigate, crashing on nearly every episode, whereas over‐weighting it caps improvement near the demonstrator's level. Network capacity has negligible impact, whereas larger rollout buffers consistently yield the best and most consistent navigation. We further find that training reward and critic explained variance are unreliable proxies for navigation quality, underscoring the need to evaluate policies on held‐out targets.

RL-based Swarm Intelligence in Autonomous Drone Navigation for Energy-Optimized Surveillance and Reconnaissance

Autonomous drone operations in large-scale and dynamic environments face significant challenges related to energy efficiency, communication reliability, and scalability. Traditional single-drone systems and centralized swarm architectures often suffer from limited adaptability, communication bottlenecks, and single points of failure. To address these limitations, this paper proposes a decentralized autonomous swarm framework integrating Deep Reinforcement Learning (DRL) for local decision-making and Federated Reinforcement Learning (FRL) for swarm-level coordination. The proposed system enables drones to collaboratively learn navigation policies while operating independently without continuous centralized control. A fuel-aware reward optimization framework is introduced to balance target detection performance and energy consumption. The framework is evaluated against classical navigation approaches including A*, Greedy, and Random baselines within multiple simulation environments developed using Python, Pygame, FastAPI, and Unity. Experimental results demonstrate that the proposed FRL-based framework achieves superior cumulative reward performance in dynamic environments while maintaining robust decentralized coordination. The results validate the effectiveness of combining DRL and FRL for scalable, adaptive, and energy-efficient autonomous drone swarm navigation.