Skip to content

Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models

Sep 2026 · arXiv.org · Vol abs/2609.32069 · 3 citations · ⚡ 1 influential · 40 references
Computer Science

TL;DR

FIND is introduced, an agentic real-world RL framework that closes the loop between scene understanding, weakness-aware practice, self-evaluation, and policy improvement in a persistent workspace and reframes autonomous practice as a scene-conditioned, performance-aware task-selection problem.

Abstract

Vision--language--action (VLA) models provide strong priors for robotic manipulation but are typically deployed as frozen policies, unable to improve from their own failures. Real-world reinforcement learning (RL) offers a path to continued improvement, yet manual environment resets and task-success supervision hinder autonomous learning. We introduce \textbf{FIND}, an agentic real-world RL framework that closes the loop between scene understanding, weakness-aware practice, self-evaluation, and policy improvement in a persistent workspace. FIND reframes autonomous practice as a scene-conditioned, performance-aware task-selection problem: instead of restoring a predefined scene after each rollout, it uses the resulting scene to determine what to practice next. A vision--language agent identifies feasible tasks from a predefined library, prioritizes those with lower recent success rates, and evaluates outcomes using paired pre- and post-execution observations. We instantiate FIND with a frozen $\pi_{0.5}$ VLA and residual off-policy RL. Across eight real-world manipulation tasks, the independent human-assessed success rate improves from $55\%$ to $71.9\%$. A representative run completes 456 autonomous episodes within 6 hours of interaction, requiring 30 scene-recovery interventions and no human-provided reward labels during online learning. Ablations and systematic evaluations further examine key design choices, agent evaluation accuracy, and human intervention requirements. Our website is made publicly available at: FIND.github.io.

View source

Similar papers

Teach and Grow: An Agent-Centered Architecture for General Robot Learning

Vision-language-action (VLA) and world-action models typically absorb unfamiliar manipulation tasks through additional robot data collection and policy optimization. This recurring retraining burden slows the acquisition of new behavior. We present Teach-and-Grow Learning (TGL), a training-free architecture that turns...

Chang Nie, Zhe Liu, He-Sheng Wang · 2 citations

Reinforcement Learning for Real-Time Vision-Language-Action Policies

This work instantiates Real-Time EXPO-FT, an RL framework for finetuning real-time VLA policies that meets the real-time control requirements of dynamic real-world manipulation, demonstrating rapid, sample-efficient adaptation to challenging real-world dynamics.

Perry Dong, Kuo-Han Hung, D. Sadigh et al. · 2 citations

Prioritized Rollouts for Efficient World Model-based Vision-Language-Action Policy Optimization

Vision-Language-Action (VLA) models have emerged as a powerful paradigm for embodied intelligence, but fine-tuning them with reinforcement learning (RL) remains constrained by the cost of real-world robot interaction. Model-based reinforcement learning (MBRL) reduces this cost by using a learned world model to generate...

Yi-Fei Sheng, Hao-Xiang Ren, Zhi-Long Zhang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

BEE: Intervention-Adaptive Real-World Reinforcement Learning with Vision-Language-Action Models

BEE is introduced, an intervention-adaptive framework for real-world RL on a frozen VLA that lets the policy go BEyond Expert imitation, and a Correction Model predicts how a human would correct a given VLA proposal and how consistent the correction is along each action dimension.

Wei-Hui Zhao, Xiao Yan, Zu-Nian Wan et al. · 0 citations
Sep 2026

In-Context Robot Learning with VLM Agents

GPT-Policy is introduced, a general-agent framework for in-context robot learning that integrates a context compiler that preserves task-relevant visual transitions, a VLM that proposes robot-tool actions, and a constrained controller that verifies and executes each action and reports its outcome.

Dong-Zhou Cheng, Taoran Yi, Ye Fang et al. · 16 citations · ⚡4

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.