HYPER-GNC, a multi-task reinforcement learning framework in which a hypernetwork maps physics-informed task embeddings to the weights of a shared actor-critic policy, enabling a single compact controller to master four distinct GNC tasks: velocity tracking, docking, inspection, and navigation with obstacle avoidance.
Abstract
Autonomous free-flying robots in orbital environments require controllers that are both versatile and resource-efficient, yet maintaining a separate, task-specific policy for each mission profile is architecturally brittle and limits operational flexibility as requirements evolve. We introduce HYPER-GNC, a multi-task reinforcement learning framework in which a hypernetwork maps physics-informed task embeddings to the weights of a shared actor-critic policy, enabling a single compact controller to master four distinct GNC tasks: velocity tracking, docking, inspection, and navigation with obstacle avoidance. The continuous embedding space allows the controller to generalize to novel mission configurations at deployment time without any retraining. Extensive experiments demonstrate that HYPER-GNC achieves sample efficiency comparable to single-task specialists while maintaining stability under significant inertial perturbations and external body wrenches. We further validate the framework on a physical satellite emulator, successfully bridging the simulation-to-reality gap across all mission profiles. Code, trained models, and deployment scripts are made publicly available to support reproducibility.
Ensuring QoS provisioning in low-altitude wireless networks requires UAV positioning and navigation strategies that adapt to dynamic environments and generalizes across heterogeneous network scenarios. This paper proposes a digital twin (DT)-assisted meta reinforcement learning framework for multi-agent UAV trajectory planning. A high-fidelity network DT serves as a supervisory layer to generate key performance indicators (KPIs) and fine-grained channel knowledge, which guides both domain-specific learning and cross-domain validation. Building on the twin-informed UAV landmarks, we then develop a weakness-aware meta learning scheme: in the inner loop, agents are trained cooperatively toward the self-discovered landmarks under dynamic conditions; in the outer loop, navigation policies are evaluated via the DT to identify bottlenecks and generate targeted hard scenarios, enabling robust adaptation across diverse scenarios. Extensive simulations show that our framework achieves up to 4× higher service coverage compared to baselines, while the target-aware outer-loop adaptation further improves cross-scene performance and model generalization.
Jiayuan Huang, E. Tucker, Ruozhou Yu et al.· International Conference on...· 0 citations
: Multi-unmanned Aerial Vehicle (multi-UAV) missions require task assignment strategies that adapt to dynamic urgency, threats, and resource constraints. Fixed heuristics lack flexibility, while end-to-end learned policies often omit explicit safety and verification mechanisms. This paper proposes a learning-guided neuro-symbolic orchestration framework in which reinforcement learning operates at a meta level to select among heterogeneous symbolic solvers within a closed simulation loop. Candidate assignments are regulated through a dual-layer verification mechanism comprising a feasibility filter and an episode-level simulator-based evaluator. The meta-decision layer is trained using a Double Deep Q-Network (Double DQN) with rewards reflecting completion, expiration, battery consumption, and threat exposure. Experiments across diverse scenarios demonstrate the effectiveness of adaptive solver selection and improved safety–efficiency trade-offs compared to fixed baselines.
Muhyun Byun, S. Doo, Eunae Lee· Proceedings of the 16th Inte...· 0 citations
Hyper-redundant robots are essential for navigation in highly constrained environments, yet their high-dimensional kinematics impose a severe computational burden on real-time motion planning. While optimization-based methods ensure tracking precision, their high computational latency makes them unsuitable for online feedback loops; conversely, geometric heuristics offer speed but lack kinematic fidelity. To resolve this efficiency-accuracy trade-off, we present an imitation learning framework tailored for path-following tasks. First, to address the instability of expert data generation caused by non-differentiable minimax objectives, we propose a refined Soft-Maximum formulation that produces smooth, kinetically consistent demonstrations. Second, we mitigate the covariate shift inherent in Behavior Cloning (BC) through a two-stage noise-injection curriculum, enabling the agent to learn robust recovery policies entirely offline without requiring an interactive expert. Finally, we design a structured policy network that effectively fuses high-dimensional path descriptors with low-dimensional proprioceptive states. Extensive simulations demonstrate that our approach achieves optimization-level accuracy with inference speeds comparable to geometric heuristics, validating its efficacy for high-precision inspection tasks.
Nailong Bu, Ningyuan Luo, Chao-yuan Liu et al.· 2026 IEEE/ASME International...· 0 citations
Cooperative navigation of multi-agent UAVs in complex environments faces key challenges including local optima traps, sparse rewards, learning imbalance among agents, and insufficient cross-scenario generalisation. This paper proposes a multi-agent deep reinforcement learning framework that addresses these issues through coordinated exploration, demonstration exploitation, safe curriculum scheduling, and structure-aware generalisation. First, a perception mechanism combining memory of visited states, directional novelty estimates, and penalty backpropagation enables agents to proactively detect and escape local optima. Second, a hierarchical collaborative demonstration buffer with tiered behaviour cloning manages trajectories by degree of team collaboration and applies differential supervision to the actor network, improving demonstration utilisation under sparse collaborative signals. Third, a safety-aware dual-condition curriculum scheduling mechanism reviews mastered scenarios through back-testing and experience pre-filling during training, suppressing catastrophic forgetting while ensuring both task performance and flight safety. For generalisation, local geometric features computed from sensor readings are abstracted into a domain parameter, through which a structure-aware gating network and mixture-of-experts mechanism condition the policy on local structural patterns rather than scenario-specific coordinates, enabling cross-scenario transfer without exposure to the target environment. The framework is further validated under mixed static-dynamic obstacle settings, showing robust adaptability to dynamic disturbances. Simulation results confirm strong performance in collaboration success rate, navigation robustness, zero-shot cross-scenario generalisation, and dynamic environment adaptability.
While Deep Reinforcement Learning (DRL) offers robust real-time control for Active Debris Removal (ADR), current literature often assumes idealized dynamics and isolated mission segments. This paper presents an end-to-end 6 Degrees of Freedom (6-DOF) Guidance, Navigation, and Control (GNC) architecture using Proximal Policy Optimization (PPO) for a complete four-phase ADR mission. Bypassing standard linear approximations, the custom environment utilizes an absolute Keplerian N-Body Earth-Centered Inertial (ECI) propagator. It enforces operational realism through continuous Tsiolkovsky propellant dynamics, unmitigated kinematic handovers between phases, and instantaneous Huygens-Steiner inertia updates upon target capture. Trained via Curriculum Learning and astrodynamic energy rewards, the agent successfully executes fuel-optimized orbital transfers, performs dynamic obstacle avoidance, stabilizes proximity operations despite handover drift, and autonomously adapts to the severely degraded mass dynamics of the composite orbital vehicle.
Federico Falconi, Kirolos Romany Anwar Kamel, D. Menegatti et al.· International Conference on...· 0 citations
The problems of partial observability and sensor shortage pose a significant challenge for autonomous Unmanned Aerial Vehicles (UAVs) as they prove to be challenging for conventional Deep Reinforcement Learning (DRL) methods to undertake well under such conditions. In this paper, a memory-augmented Proximal Policy Optimization (PPO) model extended using a Long Short-Term Memory (LSTM) network is proposed as a solution to such challenges. The observation space is constructed from 2D LiDAR and Inertial Measurement Unit (IMU) data to sense simultaneously external observation and internal state of motion, whereas the action space consists of continuous velocity commands. A shaped reward function is optimized for encouraging safe target approaching, obstacle avoidance, and convergence speed. Experimental outcomes show that the PPO-LSTM described herein achieves smoother paths, more robust reward convergence, and a much lower rate of collision than regular PPO. It also generalizes to new environments with movable obstacles. Qualitatively, the success rate increased from 64.5% to 83.9%, collision frequency reduced by over 70%, and path efficiency increased from 0.60 to 0.85, without suffering from unstable training behavior
M. Haddad, Dhayaa Khudher· Kufa journal of Engineering· 0 citations
