Skip to content

WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory

Sep 2026 · 3 citations · 100 references
Computer Science

TL;DR

WorldCrafter is a video world model that learns a camera-queryable implicit 3D-aware memory that enables streaming scene exploration from a single input image or text prompt and shows substantial gains in long-horizon consistency and camera-control accuracy while preserving visual quality during minute-scale exploration.

Abstract

Video world models enable interactive exploration of dynamic environments, yet struggle to respect prior observations over long horizons and across viewpoints. We present WorldCrafter, a video world model that learns a camera-queryable implicit 3D-aware memory for this purpose. The key insight is to let the requested viewpoint shape how multi-view evidence is compressed into the video generator's limited token budget. Trained jointly with the video generator, a memory encoder and pose-conditioned readout module integrate historical observations into a fixed set of target view-specific tokens before denoising, without explicit depth-based correspondences. By combining this memory with recent temporal context and few-step distillation, WorldCrafter enables streaming scene exploration from a single input image or text prompt. Experiments across static and dynamic scenes show substantial gains in long-horizon consistency and camera-control accuracy while preserving visual quality during minute-scale exploration.

View source

Similar papers

Preprint Aug 2026

Sekai2: From World Exploration to Interactive World Modeling

Sekai2 is introduced, a multi-source real-world video dataset that carries the world-exploration footage of Sekai toward interactive world modeling, and Corpus-scale analyses demonstrate complete pose-and-caption coverage, broad geographic and semantic diversity, varied camera trajectories, and highly non-redundant tem...

Kang He, Wen-Shuo Peng, Zi-Hui Gao et al. · 2 citations · ⚡1
Preprint Sep 2026

World in World: Explore the World with World Models

World in World is presented, a training-free inference-time interface that converts heterogeneous control evidence into camera- and time-labelled clean visual states, which are read through the native self attention of a frozen causal video model.

Chen-Xi Song, Yan-Ming Yang, Chi Zhang · 1 citation
Preprint Sep 2026

VideoTok4D: A 4D-Aware Video Tokenizer for Compact World Representation

VideoTok4D, a novel 4D-aware video tokenizer for compact world representation that achieves state-of-the-art performance while requiring up to 4 orders of magnitude less storage than dense 4D representations, and substantially shortens diffusion sequences, enabling efficient generation.

Xin-Yi Chen, Han-Xin Zhu, Xi-Jun Wang et al. · 1 citation
Preprint Sep 2026

AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video

Interactive video world models must maintain broad scene context under camera motion while producing high-fidelity observations with low latency. Existing approaches face a representation trade-off: perspective models operate on local views and must preserve off-screen content over long rollouts, whereas broader spatia...

Jia-Ming Tan, Ming-Liang Zhai, Zhen Li et al. · 2 citations
Preprint Sep 2026

Honeycomb: Constant-Size Scene Memory Representation for Video World Models

Video world models require persistent scene memory to maintain consistency during long-horizon video generation. Existing spatial memories accumulate RGB observations or latent features, increasing storage requirements as generation proceeds. We introduce Honeycomb, a video world model built on HexMemory, our proposed...

J. Shi, Kai-Chen Zhou, Haoyu Chen et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.