Skip to content
Preprint

GALA: Geometry-Aware Latent Action Modeling for Vision-Language-Action Model Pretraining across Embodiments

Sep 2026 · 1 citation · 40 references
Computer Science

TL;DR

GALA combines visual latent actions that capture scene-level dynamics with geometric latent actions that capture shared fine-grained end-effector articulation, providing effective supervision for VLA pretraining from multi-embodiment data, including action-free ego-centric human videos.

Abstract

Learning large-scale vision-language-action (VLA) models from multi-embodiment datasets remains challenging due to heterogeneous action spaces across end effectors. Although latent action models (LAMs) can learn embodiment-agnostic action representations from diverse video data, existing image-based LAMs often fail to capture fine-grained end-effector articulation, particularly finger-level geometric changes in human and dexterous robot hands. To address this limitation, we propose GALA, a Geometry-Aware Latent-Action modeling framework that augments image-based latent actions with 3D end-effector geometric motion. However, naively incorporating point clouds yields fine-grained action representations with limited shared semantics, hindering cross-embodiment pretraining. To address this issue, we introduce the Unified End-effector Motion Representation (UEMR), which preserves fine-grained motion information while improving the cross-embodiment generalizability of latent actions. Building upon UEMR, GALA combines visual latent actions that capture scene-level dynamics with geometric latent actions that capture shared fine-grained end-effector articulation, providing effective supervision for VLA pretraining from multi-embodiment data, including action-free ego-centric human videos. Experiments on fine-grained motion probing, cross-embodiment retrieval, and downstream VLA evaluation demonstrate GALA's effectiveness in modeling generalizable fine-grained motions across embodiments, achieving 68.3% RoboCasa-GR1 success rate and 75.5% real-world success rate. Code, appendix, and demos are available at https://puzhenyuan.github.io/GALA-website/.

View source

Similar papers

Preprint Aug 2026

GWM-VLA: Geometry-Aware Latent World Modeling for Vision-Language-Action Learning

GWM-VLA is proposed, a geometry-aware latent world modeling framework for VLA learning that combines geometry-aware multi-view state encoding, global context-conditioned target-view prediction, and shared latent-action representations grounded by robot-action supervision.

Yanping Zhao, Hang Yu, Yi-Wei Wang et al. · 2 citations
Preprint Sep 2026

Dream4ACT: A Shared Visual Action Interface for Multi-Embodiment Video-Action Modeling

This work introduces a shared visual action interface, called action views, which render target joint configurations from four prescribed virtual cameras using URDF-based forward kinematics, and proposes a training-free, URDF-constrained multiview recovery mechanism, without a learned embodiment-specific decoder.

Xiang-Yu Zhu, Jin Xu, Yue (Sophie) Guo et al. · 0 citations
Preprint Aug 2026

One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation

UCAG-P is introduced, a camera-centric unified action formulation that structurally aligns heterogeneous embodied datasets into a shared geometric action space, and allows a shared VLA policy to learn transferable manipulation geometry while retaining embodiment-specific controllability.

Xiaomi Embodied Intelligence Team, University of Macau Shaoqing Xu, Fang Li et al. · 0 citations
Preprint Aug 2026

Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models

This work proposes VLAct, a VLA-oriented VLM backbone trained on broad, heterogeneous, multi-embodiment robot data before task-specific fine-tuning, showing that representation-centric continued pre-training can deliver highly competitive performance under a modest compute budget and is an important independent axis of...

Sen-Qiao Yang, Cheng-Yao Wang, Yu-Xin Chen et al. · 8 citations
Preprint Aug 2026

LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models

LAWM-3D is proposed, which introduces three tightly coupled key designs: a multi-view invariant unified action tokenization scheme for learning 3D-aware latent actions, a geometric alignment constraint that anchors intermediate encoder features to a pretrained 3D foundation model, and a non-injective RGB-D joint recons...

Jia-Rui Yang, Jiale Zhange, Jiawei Li et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.