The Agentic Reward System (ARS) is introduced, an inference framework for progress reward modeling with general-purpose vision-language models (VLMs), without additional reward-model training, and its results suggest that structured inference and verification can improve the usefulness of general-purpose VLMs for robot reward modeling.
Abstract
Progress reward modeling is the problem of estimating how a robot's behavior changes task progress over time. Reliable estimation requires distinguishing meaningful state changes from failed attempts and task-irrelevant actions. We introduce the Agentic Reward System (ARS), an inference framework for progress reward modeling with general-purpose vision-language models (VLMs), without additional reward-model training. Given an offline trajectory and a task instruction, ARS uses adaptive visual inspection for both event proposal and verification. A subagent proposes a task-relevant event timeline, which a primary agent verifies and revises before estimating per-frame progress. ARS can incorporate optional terminal outcome labels and visual references to inform its judgments. It can also audit progress estimates from external reward models. We evaluate ARS with a 27B VLM on a controlled semantic-mismatch benchmark and downstream policy learning in simulation and on a real robot. The benchmark reveals that several evaluated reward baselines assign spurious progress to wrong-object manipulation even in simple pick-and-place scenes. ARS better suppresses these errors and outperforms these baselines in simulation policy learning. We further demonstrate that ARS supports long-horizon policy learning from mixed-quality offline experience on real-robot multi-screw fastening in a full-scale laboratory replica of an industrial washing-machine assembly line. These results suggest that structured inference and verification can improve the usefulness of general-purpose VLMs for robot reward modeling. Code is at https://github.com/midea-ai/ars
Recent advances in robot learning have enabled generalist control policies capable of completing a wide range of tasks. However, their performance degrades when deployed in unseen environments, making it critical to detect failures and teach recovery behaviors. Existing runtime monitoring methods often require task- an...
Intrinsic Robot Rewarding (IRR) proposes a reusable approach to learn and improve from the data and experience already available in industrial robot systems, linking reward reliability to task success and supervision effort.
This work introduces eVTA, which learns success probabilities from mixed-quality policy rollouts through temporal-difference-style bootstrapping, without expert demonstrations or intermediate annotations, and introduces RL with Evolving Rewards (RLER), a closed-loop framework that adapts eVTA using newly collected roll...
Duo Wu, Hai-Feng Wang, Rongwei Lu et al.· arXiv.org· 1 citation
Learned visual reward models are increasingly used to optimize robot policies, yet a reward model can score an execution that acts on the wrong object as highly as one that completes the task. We show that optimizing such a reward can amplify these wrong-object failures while reward and task success both rise, so the s...
Jia-Xuan Luo, Xing-Guo Xu, Shan-Shan Wang et al.· 2 citations
FIERCE is presented, a generalist-initialized reinforcement learning framework centered on a unified, task-adaptive progress-failure evaluator that separates feedback quality, policy-learning efficiency, and deployment cost across simulation and two contact-rich real tasks.
Run-Jia Tan, Yuang Tu, Yu-Jie Yan et al.· arXiv.org· 0 citations
Long-horizon LLM agents make multi-step decisions, yet outcome-reward-only reinforcement learning provides little direct supervision for individual steps. We introduce AttnSR, a step-level reward augmentation method that derives reward allocations from policy attention extracted during training replay, without a separa...
Meng-Song Wu, Tian-Chen Zhou, Zhao-Chen Su et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 9, 2026