Skip to content

World Models for Embodied Intelligence: From Plausible to Controllable to Actionable

Sep 2026 · arXiv.org · Vol abs/2609.16697 · 1 citation
Computer Science

TL;DR

This framework surveys manipulation, navigation, locomotion, autonomous driving, and general embodied learning, tracing technical progressions, clarifying capability requirements, and examining datasets, benchmarks, and evaluation protocols.

Abstract

World models connect perception and decision-making in embodied intelligence by maintaining hidden state, anticipating consequences, comparing interventions, and adapting when execution departs from expectations. Although progress is often measured by visual fidelity, their value lies in improving behavior. Before reaching for a cup, a person anticipates its weight and resistance to grasping, shaping the hand before contact. Such anticipation is coarse and rarely pictorial, yet it guides action. This raises a central question: which predictive capabilities improve behavior? Existing surveys, organized by architecture, output modality, or application domain, leave this question implicit. We introduce three progressively stronger capability levels: Plausible models preserve task-relevant temporal, geometric, or physical structure; Controllable models additionally predict how interventions alter that structure; and Actionable models translate predictions into measurable gains in planning, action, learning, evaluation, verification, recovery, or data selection. We complement this hierarchy with a 3 x 4 matrix crossing geometry, physics, and action grounding with improvement loops centered on data, rewards, policies, and the model itself. Using this framework, we survey manipulation, navigation, locomotion, autonomous driving, and general embodied learning, tracing technical progressions, clarifying capability requirements, and examining datasets, benchmarks, and evaluation protocols. We identify challenges in long-horizon consistency, uncertainty calibration, causal intervention testing, latency, verification and recovery, and cross-embodiment transfer. This perspective shifts evaluation from visual plausibility toward whether predictions capture task-relevant state, reflect intervention effects, and improve the closed-loop behavior of embodied agents.

View source

Similar papers

Sep 2026

Where Do Embodied Decisions Come From? Rethinking Latent and Explicit Reasoning

Chain-of-Thought (CoT) reasoning is increasingly incorporated into Vision-Language-Action (VLA) models, yet it can degrade the performance of stronger embodied agents. We investigate this capability-dependent effect by distinguishing explicit reasoning from latent decision computation, i.e., perception-grounded computa...

Yuan-Peng Lin, Zi-Yu Zhou, Jin-Long Zhao et al. · 0 citations
Preprint Aug 2026

XEWorld: Can Action-Conditioned World Models Generalize to Unseen Robot Embodiments?

XEWorld is introduced, a controlled cross-embodiment testbed for world models that isolates embodiments by evaluating held-out robots within physically identical scenes, highlighting that achieving true cross-embodiment generalization requires architectural innovations that decouple visual appearance from underlying ph...

Yixiang Chen, Jiabing Yang, Yuan Xu et al. · 0 citations

Achieve What You Imagined: Learning to Align Actions with Visual Plans

This work uses a frozen action-conditioned world model to predict action-conditioned consequences and construct feedback based on consistency between the two future predictions and alignment with the terminal goal to show that cross-model prediction discrepancy can provide useful feedback for improving robot policies u...

Yu-Heng Qiao, Zi-Ran Wei, Xiao-Hang Wang et al. · 0 citations
Preprint Aug 2026

WorldSimProbe: Diagnosing Simulator Faithfulness in Action-Conditioned World Models for Embodied Manipulation

This work formalizes Observable Simulator Contract, a minimal contract that any action-conditioned physical simulator should satisfy: supplied actions must induce corresponding agent motion, and environment responses must be grounded in that realized motion.

P. Co, Sichen Hu, Chun-Xuan Jiao et al. · 1 citation
Sep 2026

PSR: Predictive Sensorimotor Representation Learning for Contact-Rich Manipulation

Predictive Sensorimotor Representation learning is introduced, a framework that learns a hierarchy of predictive representations from multimodal sensorimotor signals and integrates them into the action stream of a visuomotor policy, and is instantiate within a Vision-Language-Action model, resulting in PSR-VLA.

Sheng-Bao Li, Peng Xu, Chao Tang et al. · 0 citations
#robotics Preprint Aug 2026

Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models

PILOT (Physical Inference for Latent Optimized Trajectories) bridges the gap between high-level physical condition evolution and low-level action trajectory generation within the Action Model, creating a structural bottleneck while weakening the predictive capability of world evolution modeling for action generation.

Xiangkai Ma, Yue Ma, Junjie Wang et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.