This paper proposes Bridge3D, a method that integrates both implicit and explicit 3D geometry guidance into pre-trained 2D VLA models, enabling them to see and act in 3D, and introduces two strategies that enriches visual tokens with features from 3D foundation models to improve seeing and acting in 3D.
Abstract
Vision-Language-Action (VLA) models have demonstrated remarkable generalization in robotic manipulation via large-scale multimodal pretraining. However, VLA models are mainly trained on 2D-centric observations, which inherently constrains their capacity for precise spatial manipulation. Previous methods enhance 3D awareness by introducing implicit spatial priors, but still lack explicit geometry guidance. In this paper, we propose Bridge3D that integrates both implicit and explicit 3D geometry guidance into pre-trained 2D VLA models, enabling them to''see''and''act''in 3D. Bridge3D introduces two strategies: 1) Implicit Fusion, which enriches visual tokens with features from 3D foundation models to improve''seeing''in 3D; 2) Explicit Conditioning, which integrates action denoising with an explicit 3D semantic field to achieve''acting''in 3D. Furthermore, we utilize the proposed layer-wise linear probing to improve learning efficiency. Experiments show that Bridge3D achieves superior performance against state-of-the-art methods. On the RoboTwin 2.0 benchmark, Bridge3D exceeds $\pi_0$ by 14.0 percentage points, while in real-world experiments, it outperforms Spatial Forcing by 11.7 percentage points. These results demonstrate Bridge3D's strong capabilities in high-precision and spatial-sensitive manipulation tasks.
V-Link is proposed, which explicitly recovers visual representations during the vision-language (VL) to action (A) feature transfer and injects them into Action DiT through asymmetric pathways.
Ye-Hao Lu, Jia-Rui Yang, Yu-Ning Su et al.· 0 citations
Three-dimensional perception is critical for robotic manipulation, particularly for high-precision tasks, as recovering metric depth and precise 3D object positions from monocular RGB observations is inherently ill-posed. However, many Vision-Language-Action (VLA) models rely solely on RGB observations for perception....
I. Liu, Jason Chen, Gaurav S. Sukhatme et al.· 1 citation
This work proposes FOCAL-VLA, a framework that combines subtask-guided geometry distillation with implicit world modeling to learn representations of current spatial structure and future interaction dynamics, and shows that FOCAL-VLA outperforms baselines on both simulation benchmarks and real-world manipulation tasks.
Zhi-Yuan Gao, Di Wen, Yan Zhan et al.· arXiv.org· 1 citation
Vision-language-action (VLA) models encode semantic information from vision-language pretraining, but manipulation also requires precise spatial reasoning. We present GeoBridge-VLA, a two-stage method for learning geometric features from a pretrained VLA's frozen visual encoder and using them for action prediction. Sta...
Hyunjung Song, Kangmin Kim, Loren Jinsoo Um et al.· 0 citations
MaskVLA, a masking-based fine-tuning strategy that randomly masking a small portion of the main camera's visual information leads to the emergence of robust policies, thereby enhancing the model's capability to tackle complex manipulation tasks and improving its generalization performance.
Yuxuan Jiang, Jia-Ying Huang, Ge Wang et al.· arXiv.org· 0 citations
Robo-Harness K1 is introduced, a robot-use agent (RUA) framework that exposes perception as tools that makes 3D geometry accessible without changing the VLM architecture or training a depth encoder, and suggests that perception-augmented RUAs offer a promising route to sample-efficient, generalizable robotic policies t...
Ze-Xi Li, Ye-Hang Zhang, Wen-Qian Li et al.· 6 citations· ⚡2
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.