Skip to content
Preprint

PhysWAM: Physically Consistent World Action Model for Autonomous Driving

Sep 2026 · 0 citations · 80 references
Computer Science

TL;DR

PhysWAM is presented, a unified world-action model for autonomous driving that co-denoises multiview video, metric depth, and ego motion within a single flow-matching transformer and Coupled Point Projection is introduced, demonstrating that the geometric relationship between scene depth and ego motion provides a direct way to couple world and action generation within a simple unified model.

Abstract

World-action models (WAMs) jointly predict how a scene will evolve and how an agent should act, however joint generation alone does not necessarily impose a shared geometric constraint on these predictions. We present PhysWAM, a unified world-action model for autonomous driving that co-denoises multiview video, metric depth, and ego motion within a single flow-matching transformer. To ground world and action generation in measured scene geometry, we introduce Coupled Point Projection (CPP) that unprojects the generated depth into 3D points, transforms them using the generated $\mathrm{SE}(3)$ ego motion, and minimizes their distance to LiDAR points transformed using the recorded ego motion. This geometric constraint promotes physical consistency with the measured scene by jointly supervising generated depth and motion alongside their standard flow-matching objectives. At inference, trajectory selection relies only on a simple label-free consensus rule, with no learned scorer or simulator feedback. We evaluate PhysWAM across NAVSIM v1 and v2 planning, zero-shot closed-loop transfer, and future video and metric-depth prediction. Despite PhysWAM's simple selection procedure, it achieves strong planning performance and transfers zero-shot to unseen driving environments. It also generates accurate metric depth and temporally coherent video, with CPP improving both planning and depth prediction. Together, these results demonstrate that the geometric relationship between scene depth and ego motion provides a direct way to couple world and action generation within a simple unified model.

View source

Similar papers

Preprint Aug 2026

GeoWAM: Visual Geometry World Action Models for Autonomous Driving

Geometry is established as an effective state representation for autonomous driving and geometry pretraining as a general, scalable strategy for downstream planning.

Yiren Lu, Xin Ye, Jiaming Liu et al. · 1 citation · ⚡1
Preprint Sep 2026

MM-Future: Multi-Mode Joint World-Action Modeling for Autonomous Driving

MM-Future, a world-action model that generates multiple paired scene-action hypotheses and models bidirectional interaction within each pair, shows consistent improvements over both single-mode and action-only variants, validating the benefit of multi-mode joint world-action modeling.

Shuai Liu, Hechangle Gong, Hao Jiang et al. · 1 citation
Sep 2026

MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence

MachEmbodied-U0, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture, is presented, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture that combines competitive...

Hao-Ran Wen, Wen-Fu Wang, Kun-Song Shi et al. · 1 citation
Preprint Aug 2026

GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation

GaussianDream demonstrates that training-time current Gaussian reconstruction and future Gaussian prediction provide effective 3D supervision, but its dense VGGT/TGE-based prefix jointly carries state, dynamics, and action-conditioning information.

Yu-Qing Jiang, Zi-Jian Zhang, Wei-Tao Zhou et al. · 1 citation
Preprint Aug 2026

GaussianWAM: Distilling Geometry and Semantics from 3D Gaussian Fields into World-Action Models

GaussianWAM is proposed, a training-time representation-enhancement framework that organizes geometric and semantic supervision through a 3D Gaussian field and improves performance on standard LIBERO and shows positive transfer trends on RoboTwin and real-world manipulation.

Zi-Jian Zhang, Yu-Qing Jiang, Wei-Tao Zhou et al. · 3 citations
Preprint Aug 2026

Energy-Structured Latent World Models with Neural Time Fields for Physically Constistent Open-World Motion Planning

A novel Energy-Structured Latent Latent World Model to structure the ELWM latent state to explicitly carry energy and momentum, ensuring strictly causal transitions via dissipation and control ports, and demonstrates that embedding explicit physical structures into latent spaces intrinsically bridges the gap between pr...

Yapeng Liu, Yuan-Zhao Zhai, Bo Ding et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.