Skip to content
Preprint

Dream4ACT: A Shared Visual Action Interface for Multi-Embodiment Video-Action Modeling

Sep 2026 · 0 citations
Computer Science

TL;DR

This work introduces a shared visual action interface, called action views, which render target joint configurations from four prescribed virtual cameras using URDF-based forward kinematics, and proposes a training-free, URDF-constrained multiview recovery mechanism, without a learned embodiment-specific decoder.

Abstract

Video generation models (VGMs) offer strong spatiotemporal priors for embodied observation--action modeling. However, joint-space action vectors lack explicit image-space structure and vary in dimensionality and semantics across embodiments, making it challenging to directly leverage the rich spatiotemporal priors of VGMs. End-effector visualizations provide an alternative but do not specify the full articulated configuration needed for robot execution. We present Dream4ACT, a world model built for joint video-action modeling across embodiments. To unify action representations across embodiments, we introduce a shared visual action interface, called action views, which render target joint configurations from four prescribed virtual cameras using URDF-based forward kinematics. This shared visual representation preserves embodiment-specific articulated geometry while allowing observation and action sequences to share a video autoencoder and diffusion transformer. Through masked flow-matching, our model supports forward dynamics, inverse dynamics, and joint observation--action generation within a single jointly trained model by varying which future sequences are corrupted. To recover executable action sequences from predicted action views, we propose a training-free, URDF-constrained multiview recovery mechanism, without a learned embodiment-specific decoder. Dream4ACT achieves an average success rate of 88.98\% on RoboTwin~2.0 and an overall score of 65.66 on TriWorldBench, supporting effective closed-loop manipulation and competitive action-conditioned multiview prediction through the visual action interface.

View source

Similar papers

Preprint Aug 2026

AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization

AnyWorld is proposed, a cross-embodiment world modeling framework that expands a single human interaction into diverse robot-native rollouts without paired human-robot demonstrations and enables independent recomposition of embodiment, viewpoint, and scene factors, allowing a single model to generate many robot-domain...

Cheng Chen, J. Bai, Jiacheng Wei et al. · 3 citations
Preprint Sep 2026

GALA: Geometry-Aware Latent Action Modeling for Vision-Language-Action Model Pretraining across Embodiments

GALA combines visual latent actions that capture scene-level dynamics with geometric latent actions that capture shared fine-grained end-effector articulation, providing effective supervision for VLA pretraining from multi-embodiment data, including action-free ego-centric human videos.

Yi-Chen Liu, Pu-Zhen Yuan, Xiang-Pei Zhu et al. · 1 citation
Preprint Aug 2026

One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation

UCAG-P is introduced, a camera-centric unified action formulation that structurally aligns heterogeneous embodied datasets into a shared geometric action space, and allows a shared VLA policy to learn transferable manipulation geometry while retaining embodiment-specific controllability.

Xiaomi Embodied Intelligence Team, University of Macau Shaoqing Xu, Fang Li et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence

World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support. In particular, visual dynamics, geometric structure, and executable action are commonly coupled through separate interfaces or downstream modules r...

Thinker Team Ubtech, Peng Gao, Ji-Chao Jiao et al. · 0 citations

MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies

Vision-Language-Action (VLA) models have recently incorporated world models to provide richer dynamic supervision beyond sparse action labels. However, explicitly predicting future images or videos may include control-irrelevant appearance, while guidance derived from holistic future visual representations and shared g...

Jing-Qi Wang, Yan Wang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.