A vision-language-action policy for autonomous lunar rover docking and its performance relative to a modular navigation pipeline is designed and a scalable training framework is developed that generates safety-aware expert demonstrations in a high-fidelity lunar simulation environment.
Abstract
Surface navigation on the Moon is subject to extreme illumination contrast and deployment-time perceptual disturbances. While modular navigation pipelines have long supported planetary rover navigation, robustness is compromised by measurement errors that propagate through hand-engineered components. End-to-end learned policies are an alternative approach to navigation, which maps observations directly to control actions without intermediate components. Vision-language-action models, in particular, have driven significant breakthroughs in robotics and autonomous vehicle research. By leveraging internet-scale vision-language pretraining, these models can be exploited for their commonsense reasoning and visual understanding to provide robust generalization with limited task-specific data. However, their applicability and performance have yet to be characterized for lunar surface operations. In this study, we design a vision-language-action policy for autonomous lunar rover docking and characterize its performance relative to a modular navigation pipeline. We develop a scalable training framework that generates safety-aware expert demonstrations in a high-fidelity lunar simulation environment. In closed-loop evaluations, our policy performs competitively relative to the modular baseline under perceptual disturbances unseen during training. To address the tradeoff between compute consumption and closed-loop safety, the policy adapts its execution horizon by determining how far the rover may safely travel in open loop. Adaptive-horizon execution reduces inference frequency while limiting hazard-contact rates relative to fixed horizons in the evaluated scenarios.
This work proposes a parameter-efficient approach to fine-tune a pretrained VLM for autonomous navigation using an Imperative Learning paradigm, and introduces a unified end-to-end navigation pipeline for natural-language-driven robotic control.
Sebastian Berger, Katharina Winter, Fabian B. Flohr· arXiv.org· 0 citations
Vision-and-Language Navigation (VLN) requires embodied agents to ground language in visual observations and make long-horizon decisions. Existing methods often address these demands through task-specific architectures. As multimodal large language models (MLLMs) become increasingly capable, an important question remain...
Ling-Yi Hong, Wei-Ren Cai, Jia-Yu Chen et al.· 0 citations
This work presents AeroManip-VLA, a scalable benchmark for aerial VLA data generation and policy evaluation, and evaluates a range of imitation learning and VLA baselines across different task settings, revealing their performance characteristics and failure modes.
Rui Huang, Yan-Lin Mu, Li-Dong Li et al.· arXiv.org· 0 citations
Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart warehouses remains largely unexplored, hindered by the lack of benchmarks that jointly provide continuous low-level flig...
Thinh D. Le, Son Nguyen, Duong Q. Nguyen et al.· 0 citations
By harnessing the inherent commonsense capabilities of pretrained Large Language Models (LLMs) and Vision-Language Models (VLMs), current Vision-Language Action(VLA) models for legged robots excel at interpreting language-based navigation commands but lack the physical grounding and explicit safety assurance required f...
A. M. Kulkarni, Saimouli Katragadda, Yu-Ze Du et al.· 0 citations
CPNav is introduced, a navigation framework that combines collision-aware planning with statistically calibrated VLM action scoring for map-free object-goal navigation and provides finite-sample guarantees at a user-specified risk level.
Qi-Zhao Chen, Yuan-Hong Zeng, Shoh Nishino et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.