Skip to content
Preprint

Knowing When to Stop Thinking: A Dissection of Expected Free Energy as a Test-Time Compute Controller for Latent World-Model Planning

Oct 2026 · 0 citations · 22 references
Computer Science

TL;DR

This work studies when expected free energy (EFE) can help decide how much computation to spend on a plan, and an EFE-inspired objective combines goal distance with disagreement among stochastic predictions.

Abstract

A planner using a learned world model can improve its predicted outcome while choosing a worse action. We study when expected free energy (EFE) can help decide how much computation to spend on a plan. An EFE-inspired objective combines goal distance with disagreement among stochastic predictions. We test candidate scoring, stopping, and online predictor updates separately. Stopping when elite cost stalls reduces refinement on tasks whose success plateaus with depth, with mean success close to fixed-depth references. At the larger tested weights, rewarding disagreement lowers success with a frozen model. Penalizing it helps at some task and weight settings but harms others. When training data omit a region needed to reach the goal, online updates recover part of the resulting loss. The tested cost-stall rule can fail when further optimization exploits model errors, because predicted cost keeps falling after task success peaks. Cost progress can guide computation on a plateau, but protecting performance after a peak requires information about the consequences of the chosen actions.

View source

Similar papers

Preprint Oct 2026

Adapt for Planning, Not just Prediction: Optimistic Test-Time World Model Adaptation

Optimistic Test-time adaptation of world models for planning benefits from an optimistic, task-aware term in addition to the prediction loss, which is inspired by reward-biased maximum likelihood from adaptive control.

Akshay Mete, D. Kalathil, P. R. Kumar · 0 citations
#machine learning Preprint Oct 2026

When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models

The component that dominates a latent world model's prediction error need not be the one whose repair most improves action selection. We show this by comparing action sequences from identical physical starts and separating endpoint error into a candidate-pool center and action-relative responses. Across four model fami...

Rui Min, Xian-Yao Li, Fang Xu et al. · 1 citation
Preprint Oct 2026

pico-WAM: Evaluating Counterfactual Prediction and Planning Error in a Compact World Action Model

When a World Action Model fails at control, is the bottleneck proposal generation or trajectory scoring? We introduce pico-WAM, a compact model with supplied physical state for tracing policy-dynamics interactions. Using 206 Push-T demonstrations and 15 training seeds, we compare data regimes and search configurations....

Keivalya Pandya · 0 citations

Aim Short to Reach Far: Your Frozen World Model Can Plan Better Than You Think

This work introduces Anchored Planning, which retrieves a recorded segment whose start and end resemble the current and goal observations, then aims at an observation shortly after its start, and without additional training, planning toward observed targets outperforms the LeWM planner on every task in the authors' lon...

Xvyuan Liu, Jian-Jie Fang, Wei Wu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Action-Consequence Alignment for Reliable Planning and Self-Improving in Latent World Models

Action Consequence Alignment (ACA), a training objective that complements forward prediction by penalizing the prediction error advantage of locally searched alternatives over factual actions without additional model components or environment interactions during training, is introduced.

Jin-Ping Wang, Zhi-Qiang Gao, Xian-Tong Zhen et al. · 1 citation
Preprint Oct 2026

From Outcomes to Steps: Reliable Success-Aware Critic Learning for Long-Horizon Language Agents

Reinforcement learning for long-horizon language agents requires identifying whether intermediate actions make eventual task success more or less likely when supervision is available only at trajectory level. The main difficulty is that a critic with aggregate return-prediction accuracy may still fail to resolve the lo...

Jia-Xi Liu, Shi-Xiong Kai, Zhen-Tao Tang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.