Skip to content

ARS: Agentic Reward System for Robot Learning

Sep 2026 · 0 citations · 47 references
Computer Science

TL;DR

The Agentic Reward System (ARS) is introduced, an inference framework for progress reward modeling with general-purpose vision-language models (VLMs), without additional reward-model training, and its results suggest that structured inference and verification can improve the usefulness of general-purpose VLMs for robot reward modeling.

Abstract

Progress reward modeling is the problem of estimating how a robot's behavior changes task progress over time. Reliable estimation requires distinguishing meaningful state changes from failed attempts and task-irrelevant actions. We introduce the Agentic Reward System (ARS), an inference framework for progress reward modeling with general-purpose vision-language models (VLMs), without additional reward-model training. Given an offline trajectory and a task instruction, ARS uses adaptive visual inspection for both event proposal and verification. A subagent proposes a task-relevant event timeline, which a primary agent verifies and revises before estimating per-frame progress. ARS can incorporate optional terminal outcome labels and visual references to inform its judgments. It can also audit progress estimates from external reward models. We evaluate ARS with a 27B VLM on a controlled semantic-mismatch benchmark and downstream policy learning in simulation and on a real robot. The benchmark reveals that several evaluated reward baselines assign spurious progress to wrong-object manipulation even in simple pick-and-place scenes. ARS better suppresses these errors and outperforms these baselines in simulation policy learning. We further demonstrate that ARS supports long-horizon policy learning from mixed-quality offline experience on real-robot multi-screw fastening in a full-scale laboratory replica of an industrial washing-machine assembly line. These results suggest that structured inference and verification can improve the usefulness of general-purpose VLMs for robot reward modeling. Code is at https://github.com/midea-ai/ars

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Reward-DAgger: Robot-Gated Interactive Imitation Learning with General-Purpose Progress-Based Reward Models

Recent advances in robot learning have enabled generalist control policies capable of completing a wide range of tasks. However, their performance degrades when deployed in unseen environments, making it critical to detect failures and teach recovery behaviors. Existing runtime monitoring methods often require task- an...

Ryan Li, Yigit Korkmaz, Erdem Biyik · 1 citation

Intrinsic Robot Rewarding: Reusing VLA Representations for Autonomous Evaluation and Policy Improvement

Intrinsic Robot Rewarding (IRR) proposes a reusable approach to learn and improve from the data and experience already available in industrial robot systems, linking reward reliability to task success and supervision effort.

Tobias Schäffer, Mohab Elkhayat, Daniela Nicklas et al. · 1 citation

Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policies

This work introduces eVTA, which learns success probabilities from mixed-quality policy rollouts through temporal-difference-style bootstrapping, without expert demonstrations or intermediate annotations, and introduces RL with Evolving Rewards (RLER), a closed-loop framework that adapts eVTA using newly collected roll...

Duo Wu, Hai-Feng Wang, Rongwei Lu et al. · 1 citation
#artificial intelligence Preprint Oct 2026

CriticHack: Evaluating Visual Rewards Under Robot Policy Optimization

Learned visual reward models are increasingly used to optimize robot policies, yet a reward model can score an execution that acts on the wrong object as highly as one that completes the task. We show that optimizing such a reward can amplify these wrong-object failures while reward and task success both rise, so the s...

Jia-Xuan Luo, Xing-Guo Xu, Shan-Shan Wang et al. · 2 citations
Sep 2026

FIERCE: From Generalist Robot Policies to Fast Specialists via Progress-Failure Feedback

FIERCE is presented, a generalist-initialized reinforcement learning framework centered on a unified, task-adaptive progress-failure evaluator that separates feedback quality, policy-learning efficiency, and deployment cost across simulation and two contact-rich real tasks.

Run-Jia Tan, Yuang Tu, Yu-Jie Yan et al. · 0 citations
#artificial intelligence Preprint Nov 2026

AttnSR: Attention Step Reward for Long-Horizon Agentic RL

Long-horizon LLM agents make multi-step decisions, yet outcome-reward-only reinforcement learning provides little direct supervision for individual steps. We introduce AttnSR, a step-level reward augmentation method that derives reward allocations from policy attention extracted during training replay, without a separa...

Meng-Song Wu, Tian-Chen Zhou, Zhao-Chen Su et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.