A test-time framework that combines multi-branch predictive analysis with experience-based adaptation, inspired by Model Predictive Path Integral (MPPI) control, and demonstrates the strong overall performance and the effectiveness of experience-guided predictive control.
Abstract
Large language model (LLM) agents are increasingly expected to interact with environments and improve their decisions during deployment. However, how anticipating future outcomes and drawing on accumulated interaction experience can jointly support decision-making remains underexplored. We propose a test-time framework, inspired by Model Predictive Path Integral (MPPI) control, that combines multi-branch predictive analysis with experience-based adaptation. At each step, the agent proposes multiple candidate actions and imagines their outcomes with a world model, so that alternatives can be compared without being executed in the environment. Since language environments provide neither explicit dynamics nor costs, the predicted outcomes are evaluated for task progress and risk, and MPPI-style weights over the resulting costs guide the final decision. A dual-branch experience memory enables self-improvement: the planner branch refines candidate generation and action selection, while the world model branch improves next-observation prediction. Experiments on ALFWorld and ScienceWorld across three backbones demonstrate the strong overall performance of our method and the effectiveness of experience-guided predictive control. Our code will be made publicly available.
Executable environments enable LLM agents to learn from the consequences of their actions. For embodied agents, those consequences extend beyond whether the current task succeeds: completing a delivery can consume the time, energy, or money needed for later work. Learning to plan therefore requires environments that pr...
With external tools, large language models (LLMs) can tackle complex tasks through sustained environmental interaction. LLM-based agents are therefore increasingly used to tackle complex long-horizon tasks in real-world environments. However, these tasks involve sequential, interdependent decisions and changes in envir...
This work proposes a steering agent that supervises execution on the fly and steers the system back on track when drift occurs, and jointly train the two paths with reinforcement learning, combining a pairwise ranking objective with group-relative policy optimization (GRPO).
Le-Yang Shen, Ya-Xin Gong, Yang Zhang et al.· 0 citations
Optimistic Test-time adaptation of world models for planning benefits from an optimistic, task-aware term in addition to the prediction loss, which is inspired by reward-biased maximum likelihood from adaptive control.
Akshay Mete, D. Kalathil, P. R. Kumar· 0 citations
Frontier black-box Large Language Model (LLM) agents are proficient at long-horizon reasoning and precise action generation. However, when it comes to a specific agentic task, they have no internalized knowledge of how that particular environment unfolds. A world model offers a natural remedy, providing the agent with...
Jiong-Xiao Wang, Han Ding, You-Zhi Luo et al.· 0 citations
Reinforcement learning provides a direct mechanism for optimizing task objectives in large language model-driven agents, but task rewards are typically sparse trajectory-level signals that only evaluate task success, without directly supervising action-conditioned environment transitions, leaving the model with limited...
Mao-Lin Ran, Min-Da Chen, Di Huang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.