Skip to content

Beyond Prediction: Steering VLM Agents with Retrospective World Modeling

Sep 2026 · 0 citations · 62 references
Computer Science

TL;DR

Retrospective World Modeling is introduced, a new agent learning paradigm that enables agents to reason backward by estimating the retrospective attribution distribution for the action that most likely caused a given transition, and an intrinsic signal that measures the probabilistic consistency between the policy action and the retrospective explanation.

Abstract

Equipping VLM agents with world modeling capabilities has shown strong potential for complex reasoning and long-horizon planning, while reducing the dependence of policy learning on costly real-world interactions. Existing methods mainly rely on prospective simulation to predict the consequences of candidate actions. However, this forward-only paradigm focuses on what will happen next and provides limited constraints for verifying whether an action is causally consistent with the observed state transition, which can lead to plausible-looking but physically incoherent behaviors. In this paper, we challenge the view of world modeling as only prospective prediction and introduce Retrospective World Modeling, a new agent learning paradigm that enables agents to reason backward by estimating the retrospective attribution distribution $P(\hat{a}{t}|s_t, s{t+1})$ for the action that most likely caused a given transition. Based on this capability, we formulate the Self-Consistency Reward (SCR), an intrinsic signal that measures the probabilistic consistency between the policy action and the retrospective explanation. Integrating SCR into reinforcement learning provides dense transition-level feedback and steers agents toward behaviors that are both task-effective and physically grounded. Extensive experiments across diverse agentic tasks show that our method substantially improves policy robustness and generalization over prospective-only world modeling baselines.

View source

Similar papers

Preprint Sep 2026

Safe Task Planning with Long-Term Graph Memory for Embodied Agents

A novel safe task-planning framework, SafeMem, which constructs and maintains a long-term semantic graph memory of the open and dynamic environment, and substantially improves safe success rates compared to state-of-the-art VLM-driven task planners.

Si-Yuan Li, Tai-Yan Lang, Ao Yan et al. · 1 citation

Prioritized Rollouts for Efficient World Model-based Vision-Language-Action Policy Optimization

Vision-Language-Action (VLA) models have emerged as a powerful paradigm for embodied intelligence, but fine-tuning them with reinforcement learning (RL) remains constrained by the cost of real-world robot interaction. Model-based reinforcement learning (MBRL) reduces this cost by using a learned world model to generate...

Yi-Fei Sheng, Hao-Xiang Ren, Zhi-Long Zhang et al. · 0 citations

DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculum

Executable environments enable LLM agents to learn from the consequences of their actions. For embodied agents, those consequences extend beyond whether the current task succeeds: completing a delivery can consume the time, energy, or money needed for later work. Learning to plan therefore requires environments that pr...

Haoqiang Kang, Yi-Ming Zhang, Yi-Yang Guo et al. · 2 citations
#reinforcement learning Review Open access Sep 2026

A survey of world models for physical AI with uncertainty representation and control

A comprehensive and technically grounded review of learning-based world models for Physical AI, with particular emphasis on closed-loop decision-making, and outlines principled directions for building reliable and scalable Physical AI systems.

Sven Kirchner, Nils Purschke, Alois Knoll · 0 citations
#artificial intelligence Preprint Oct 2026

EpicWorldModel: Exploration-driven Planning with Latent World Models

Latent world models based on Joint-Embedding Predictive Architecture (JEPA) are deterministic by design. While successful in fully observable scenarios, this paradigm breaks down when past observations and actions lead to multiple plausible future possibilities, e.g., due to occlusion. We introduce EpicWorldModel, a fr...

Bo-Wen Feng, Julian Ost, May Mei et al. · 0 citations
Preprint Oct 2026

ForeAct3D: Policy-Grounded Future World Modeling for VLA Policies

Robots need to anticipate how their actions will change the world, since manipulation success hinges on the resulting contacts and object motions. However, existing Vision-Language-Action (VLA) policies that predict future observations from shared features leave the forecast decoupled from the actions the policy will a...

Zhe Tao, Fei-Ran Wang, Gao-Wen Liu et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.