Skip to content

From World Models to World Action Models: Rethinking Next-State Prediction

Sep 2026 · arXiv.org · Vol abs/2609.34414 · 1 citation · 48 references
Computer Science

TL;DR

CF-WAM is proposed, a dynamic next-state prediction framework that samples visual, semantic, geometric, and interaction projections of the same future, standardizes them into a common video form, and supervises a unified WAM across these projections.

Abstract

Predicting the next state is a core paradigm of World Models for modeling physical dynamics, emphasizing prediction fidelity. As World Models evolve into World-Action Models (WAMs), existing methods still fix the next state before training as RGB, a single latent feature, or a static combination of predefined targets, thereby constraining action learning to the inductive biases preserved by a particular representation. To address this limitation, we propose CF-WAM, a dynamic next-state prediction framework that samples visual, semantic, geometric, and interaction projections of the same future, standardizes them into a common video form, and supervises a unified WAM across these projections. The action-relevant constraints exposed by these projections accumulate across training steps, forcing WAM to capture the underlying state-transition structure that supports multiple projections of the same action-conditioned future. This dynamic mechanism also provides a natural cross-embodiment dynamics reference frame for Human and Robot learning. By jointly learning across different next-state parameterizations, heterogeneous Human and Robot experience can bypass appearance differences and directly contribute to shared state-transition learning, improving cross-embodiment generalization. Experiments show that CF-WAM improves both training efficiency and final control performance, while translating Human experience effectively into policy gains. CF-WAM achieves state-of-the-art performance on RoboCasa-GR1 with an average success rate of 82.50%, while reaching 82.65% on LIBERO-Plus and up to 84.00% in real-world evaluations.

View source

Similar papers

#robotics Preprint Aug 2026

Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models

PILOT (Physical Inference for Latent Optimized Trajectories) bridges the gap between high-level physical condition evolution and low-level action trajectory generation within the Action Model, creating a structural bottleneck while weakening the predictive capability of world evolution modeling for action generation.

Xiangkai Ma, Yue Ma, Junjie Wang et al. · 0 citations
Preprint Aug 2026

Foresight Without Seeing: Latent Futures for World Action Models

ForeWAM is proposed, a dynamics-conditioned direct-policy WAM that provides predictive context for action generation without decoding future videos, and demonstrates that direct-policy WAMs can retain efficient action prediction while exposing predictive dynamics to the action pathway without explicitly generating futu...

Jiakai Huang, Zhongbo Wu, Siyu Xu et al. · 3 citations
Preprint Aug 2026

JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling

JEPA-WAM, a latent WAM built in a pretrained V-JEPA space, which couples latent transition prediction with continuous action generation through a shared predictor, predicts a spatially structured joint current-future target that captures task-shared visual temporal structure between current and future observations, whi...

Yi-Han Lin, Jia-Wei He, Shi-Feng Bao et al. · 12 citations · ⚡1
Preprint Sep 2026

Magic-W0: A Structured World-Action Foundation Model for Physical Intelligence

World-action models (WAMs) augment robot policies with action-conditioned environment dynamics, yet existing approaches largely rely on future observation reconstruction or generic latent prediction and lack structured, control-oriented world representations tightly coupled with action generation. We introduce Magic-W0...

Xu-Hua Chen, Zhen-Han Yin, Yu Zhang et al. · 0 citations
Sep 2026

The Right Future for Action: Learning Action-Relevant Predictive States in World Action Models

These findings motivate Action-Relevant Predictive States (ARPS), a compact predictive interface between the video and action experts that achieves 99.2% success on LIBERO and transfers to LIBERO-Plus without adaptation, reaching 87.3% and exceeding Fast-WAM by 39.2 percentage points.

Qi-Wen Gu, Ji-Fan Li, Bing-Jie Gao et al. · 8 citations · ⚡1
Sep 2026

Rethinking Representations for World-Action Modeling

World-action models jointly learn robot policies and predict future observations, making the representation space an interface between control and prediction. We study the design of this space through controlled comparisons, finding that neither reconstruction fidelity nor pre-trained perceptual features alone ensure e...

Hao-Yi Jiang, Liu Liu, Xin-Jiang Wang et al. · 5 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.