GALA combines visual latent actions that capture scene-level dynamics with geometric latent actions that capture shared fine-grained end-effector articulation, providing effective supervision for VLA pretraining from multi-embodiment data, including action-free ego-centric human videos.
Abstract
Learning large-scale vision-language-action (VLA) models from multi-embodiment datasets remains challenging due to heterogeneous action spaces across end effectors. Although latent action models (LAMs) can learn embodiment-agnostic action representations from diverse video data, existing image-based LAMs often fail to capture fine-grained end-effector articulation, particularly finger-level geometric changes in human and dexterous robot hands. To address this limitation, we propose GALA, a Geometry-Aware Latent-Action modeling framework that augments image-based latent actions with 3D end-effector geometric motion. However, naively incorporating point clouds yields fine-grained action representations with limited shared semantics, hindering cross-embodiment pretraining. To address this issue, we introduce the Unified End-effector Motion Representation (UEMR), which preserves fine-grained motion information while improving the cross-embodiment generalizability of latent actions. Building upon UEMR, GALA combines visual latent actions that capture scene-level dynamics with geometric latent actions that capture shared fine-grained end-effector articulation, providing effective supervision for VLA pretraining from multi-embodiment data, including action-free ego-centric human videos. Experiments on fine-grained motion probing, cross-embodiment retrieval, and downstream VLA evaluation demonstrate GALA's effectiveness in modeling generalizable fine-grained motions across embodiments, achieving 68.3% RoboCasa-GR1 success rate and 75.5% real-world success rate. Code, appendix, and demos are available at https://puzhenyuan.github.io/GALA-website/.
GWM-VLA is proposed, a geometry-aware latent world modeling framework for VLA learning that combines geometry-aware multi-view state encoding, global context-conditioned target-view prediction, and shared latent-action representations grounded by robot-action supervision.
Yanping Zhao, Hang Yu, Yi-Wei Wang et al.· 2 citations
This work introduces a shared visual action interface, called action views, which render target joint configurations from four prescribed virtual cameras using URDF-based forward kinematics, and proposes a training-free, URDF-constrained multiview recovery mechanism, without a learned embodiment-specific decoder.
Xiang-Yu Zhu, Jin Xu, Yue (Sophie) Guo et al.· 0 citations
UCAG-P is introduced, a camera-centric unified action formulation that structurally aligns heterogeneous embodied datasets into a shared geometric action space, and allows a shared VLA policy to learn transferable manipulation geometry while retaining embodiment-specific controllability.
Xiaomi Embodied Intelligence Team, University of Macau Shaoqing Xu, Fang Li et al.· 0 citations
This work proposes VLAct, a VLA-oriented VLM backbone trained on broad, heterogeneous, multi-embodiment robot data before task-specific fine-tuning, showing that representation-centric continued pre-training can deliver highly competitive performance under a modest compute budget and is an important independent axis of...
Sen-Qiao Yang, Cheng-Yao Wang, Yu-Xin Chen et al.· 8 citations
LAWM-3D is proposed, which introduces three tightly coupled key designs: a multi-view invariant unified action tokenization scheme for learning 3D-aware latent actions, a geometric alignment constraint that anchors intermediate encoder features to a pretrained 3D foundation model, and a non-injective RGB-D joint recons...
Jia-Rui Yang, Jiale Zhange, Jiawei Li et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.