Skip to content
Preprint

RefineFly: Failure-Aware Post-Training for Aerial Vision-Language Navigation

Aug 2026 · 0 citations · 28 references
Computer Science

TL;DR

RefineFly, a failure-aware RL post-training framework for end-to-end UAV-VLA policies, built on token-level proximal policy optimization (PPO), maintains dynamic failure memory within a two-stage scene curriculum to sustain learning from unresolved tasks as training shifts from empirical to balanced scene sampling.

Abstract

Unmanned aerial vehicle vision-language navigation (UAV-VLN) requires agents to translate visual observations and language instructions into reliable flight actions in complex environments. Although recent end-to-end UAV vision-language-action (UAV-VLA) policies reduce reliance on separately designed perception, planning, and control modules, their behavior-cloning objectives provide limited supervision for errors arising during closed-loop execution. Reinforcement learning (RL) offers a promising solution, while limited interaction budgets, long-tailed scene distributions, and policy drift complicate sustained learning. To this end, we propose RefineFly, a failure-aware RL post-training framework for end-to-end UAV-VLA policies. Built on token-level proximal policy optimization (PPO), RefineFly maintains dynamic failure memory within a two-stage scene curriculum to sustain learning from unresolved tasks as training shifts from empirical to balanced scene sampling. Throughout this process, stage-specific reference regularization constrains policy deviation. Experiments on the TravelUAV benchmark demonstrate that RefineFly outperforms all comparison methods across all three splits, improving success rate over AerialVLA by 3.12 to 8.37 percentage points with a total rollout budget of about 30\% of the training-set size. Moreover, ablations reveal that the benefits of sustained failure learning vary across evaluation splits, while the two-stage scene curriculum improves overall performance, highlighting the importance of training distributions when learning from failures.

View source

Similar papers

Preprint Aug 2026

From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation

This work presents an instruction-grounded semantic enhancement module that injects object-level semantics and relative spatial cues into the current observation state, and develops a relevance-aware dynamic temporal aggregation strategy that reweights the full history buffer while converting a few high-relevance frame...

Ze-Yuan Ma, Jiaxin Chen, Di Huang · 1 citation
Preprint Aug 2026

PILOT: Privileged Imitation Learning for End-to-End Motion Planning of Autonomous UAVs under Partial Observability

PILOT, a constraint-aware privileged imitation learning framework for vision-based end-to-end UAV motion planning under partial observability, is presented, demonstrating the practical feasibility and cross-domain generalization of the planner.

Qing-Rui Zhang, Feng Xue, Xiang Zhou et al. · 0 citations
Sep 2026

ForeFly: A Dual-Horizon World Action Model for Aerial Vision-Language Navigation

Foresight-Guided Action Refinement (FGAR), which asymmetrically exploits proximal foresight for local action enhancement and route-critical foresight for feature-wise correction and route-level guidance, is introduced, validating the effectiveness of dual-horizon foresight and FGAR learning.

Kun-Hui Wang, Xin-Tong Zhang, Junyu Gao et al. · 0 citations
Sep 2026

"Dear LLaVA, Please Drive": A Depth-Aware Vision-Language Agent for Closed-Loop Robotic Control

This work proposes a parameter-efficient approach to fine-tune a pretrained VLM for autonomous navigation using an Imperative Learning paradigm, and introduces a unified end-to-end navigation pipeline for natural-language-driven robotic control.

Sebastian Berger, Katharina Winter, Fabian B. Flohr · 0 citations

AeroManip-VLA: Scalable Vision-Language-Action Learning for Aerial Manipulation with RL-Generated Demonstrations

This work presents AeroManip-VLA, a scalable benchmark for aerial VLA data generation and policy evaluation, and evaluates a range of imitation learning and VLA baselines across different task settings, revealing their performance characteristics and failure modes.

Rui Huang, Yan-Lin Mu, Li-Dong Li et al. · 0 citations
Sep 2026

VLM-MPPI: Grounding Natural Language in Behaviorally Diverse Trajectories for Aerial Navigation

A hierarchical UAV navigation framework that aligns natural-language intent with dynamically feasible flight behaviors in cluttered indoor environments is presented, and a parallelized ensemble of six behavior-conditioned Model Predictive Path Integral (MPPI) planners are employed.

Hanbing Zhang, Fang-Guo Zhao, Ze-Rui Li et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.