Skip to content

Scaling Video Generation for Reasoning: At What Cost?

Sep 2026 · 0 citations · 37 references
Computer Science

TL;DR

Symbolic state supervision raises the 20M model's frame accuracy from 31.1% to 67.3% at the same training-data budget, suggesting that learning representations of state changes can complement scaling.

Abstract

We study whether scaling video generation enables models to reason about hidden information from the past frames, and at what computational cost. Our controlled benchmark requires predicting nine prescribed moves of an initially solved 2x2x2 Rubik's Cube from a fixed view of three faces. Correct predictions require inferring how actions change hidden states, and the simulator provides exact ground truth for evaluation. Models learn plausible cube geometry early, while correct sticker configurations require substantially more training. Although validation MSE follows approximate power-law scaling, lower MSE loss does not reliably indicate downstream reasoning capabilities. Smaller autoregressive models achieve higher state accuracy with limited compute, while larger models reach higher accuracy after more training. At roughly 0.1 PF-days, the 70M-parameter model correctly predicts the visible sticker configuration in 44.6% of post-action frames, compared with 0.3% for the 1B model, which reaches 83.7% at 3.14 PF-days. Symbolic state supervision raises the 20M model's frame accuracy from 31.1% to 67.3% at the same training-data budget, suggesting that learning representations of state changes can complement scaling.

View source

Similar papers

Preprint Aug 2026

VGI-Bench: Probing Visual Intelligence in Video Generation Models

VGI-bench is introduced, containing 27 tasks and 810 instances, organized by a two-level taxonomy of task domains and skill tags for fine-grained evaluation of visual reasoning capabilities of video generation models, and it is hoped VGI-bench will help stimulate the development of next-generation video generation mode...

Xuan He, Cong Wei, Yuhao Cheng et al. · 3 citations
Sep 2026

What Makes World Action Models Generalize? An Empirical Study of Test-Time Future Modeling

Simple-WAM is proposed, which simplifies future modeling into a single forward pass of fully noised video tokens and adapts the training-time noise schedule to this inference behavior, leading explicit WAMs in generalization performance with efficiency comparable to Latent WAMs.

Ren-Ping Zhou, Zan-Lin Ni, Zi-Hao Fan et al. · 0 citations
#artificial intelligence Preprint Aug 2026

PAWBench: How Far Are We from Probabilistically Aligned World Modeling?

This work formalizes probabilistic alignment as a distributional criterion for world models and introduces PAWBench, a benchmark for evaluating video generators as stochastic samplers of world dynamics, and introduces PAWEval, an outcome-level protocol that converts repeated video rollouts into empirical distributions...

Yuan-Dong Pu, Le Zhuo, Sayak Paul et al. · 0 citations
#artificial intelligence Preprint Sep 2026

DeepJEPA: Scaling World Models from Within

DeepJEPA is introduced, a weight-tied joint-embedding predictive world model that treats transition depth as an inner test-time scaling axis and learns when another recurrent update is worth computing for each candidate and rollout step.

Zi-Jian Jin, Yun-Bei Zhang, Yuan-Zhe Liu et al. · 1 citation
#artificial intelligence Conference Sep 2026

From Evaluation to Enhancement: Benchmarking and Improving Think-with-Video Reasoning for Video Generative Models

This work proposes Vid-PRE (Video Prompt Reasoner and Enhancer), a model-agnostic prompt rewriter that offloads the cognitive burden of reasoning to a dedicated VLM, and introduces VWG-Bench, a comprehensive benchmark spanning 9 reasoning dimensions and 38 fine-grained tasks.

Meng Luo, Yi-Chen Liu, Jia-Hao Wang et al. · 1 citation

Related blog posts

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.