Skip to content

NavGen: Visual Generative Models as a Scalable Data Engine for Embodied 3D Navigation

Sep 2026 · 2 citations · 36 references
Computer Science

TL;DR

NavGen is introduced, a text-to-video data generation pipeline that produces diverse vision-language navigation episodes across indoor and outdoor scenes and a style-diversification method that scales up long-tail data that is difficult and costly to collect.

Abstract

General-purpose robot models increasingly rely on large and diverse datasets. For embodied 3D navigation, however, existing data sources face a fundamental trade-off: simulated data can be generated at scale but often suffer from the visual sim-to-real gap, whereas real-world flight data provide realistic observations but are costly to collect. This paper studies another direction: the use of high-fidelity visual generative models as scalable data engines for embodied 3D navigation. We introduce NavGen, a text-to-video data generation pipeline that produces diverse vision-language navigation (VLN) episodes across indoor and outdoor scenes. We also propose a style-diversification method that scales up long-tail data that are difficult and costly to collect. The resulting dataset contains approximately 400K navigation episodes. We evaluate our dataset against existing UAV navigation datasets across multiple metrics, and find that the model trained on our data generally improves with scale, outperforming those trained on existing datasets. To validate real-world transferability, we deploy the trained model in world-action-model paradigm to real-world flying experiments. The final model achieves a 75\% success rate across different navigation tasks and environments.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

DepthWorld: 3D World Model for Robot Manipulation

This work introduces a calibration pipeline that combines learned stereo depth with a joint factor graph, pooling all episodes collected from the same physical robot to recover its shared kinematic parameters alongside per-scene extrinsics, and trains DepthWorld, a Stable Video Diffusion-based world model that jointly...

Jai Bardhan, Josef Sivic, Vladimír Petrík · 2 citations
Open access Aug 2026

DevGRU: Depth-Guided Visual Navigation Using a Collision-Aware Recurrent Model

The proposed DevGRU navigation system employs an action predictor that generates collision-aware future trajectories, enabling effective avoidance of immediate obstacles and has a relatively small number of trainable parameters, resulting in the fastest inference time among the baselines.

Kyung Min Han, Eunsom Kim, Young J. Kim · 0 citations
Open access Sep 2026

Open-Set 3D Scene Graphs for Field Robotics: An Outdoor Case Study

Outdoor 3DSGs can maintain compact and relatively consistent large-scale environment representations, while highlighting open challenges in handling multiple semantic modes, incorporating traversability into graph structures, and improving higher-level region understanding.

Chad R. Samuelson, Gabriel R. Slade, Joshua G. Mangelson · 0 citations
#artificial intelligence Preprint Oct 2026

EnvDreamer: Large-Scale Multimodal-to-Environment Generation for Embodied AI

Large datasets and high capacity models have accelerated progress in vision and language. This work introduces a platform aimed at bringing comparable gains to embodied learning, world models, and robotics. We present EnvDreamer, a framework that uses large language and vision language models to generate Unreal Engine...

Kabir Swain, Si-Jie Han, Antonio Torralba · 0 citations
Sep 2026

MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence

MachEmbodied-U0, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture, is presented, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture that combines competitive...

Hao-Ran Wen, Wen-Fu Wang, Kun-Song Shi et al. · 1 citation
Preprint Aug 2026

Exposing the Long-tail in Embodied Urban Navigation via Scalable Learning from In-the-Wild Videos

Learning embodied urban navigation policies from real-world data is constrained by the cost of task-specific data collection and the limited coverage of rare yet safety-critical scenarios. To address these challenges, we present a scalable framework for learning point-goal urban navigation from web-scale in-the-wild eg...

Bingyi Xia, Han Bao, Zhe-Wei Chen et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.