Skip to content

Recursive Harness Distillation across Agents for Robot Manipulation

Sep 2026 · 1 citation · 31 references
Computer Science

TL;DR

The results demonstrate the feasibility of harness distillation for robotics: intervention experience can be accumulated, refined through execution, and reused across agents to improve manipulation.

Abstract

A central goal in robotics is to enable manipulation across changing tasks and environments. Vision-language-action (VLA) models provide broad manipulation capabilities but can struggle when execution requires diagnosing failures and adapting behavior. Strong agents can discover effective interventions through interaction with these policies. We propose Recursive Harness Distillation to accumulate this experience as reusable guidance across agents. A strong agent distills its experience into a playbook for a light agent, then recursively refines the playbook using the light agent's execution feedback. The resulting playbook enables agents to reuse accumulated intervention knowledge in new task instances without updating model parameters. In real-world manipulation, the harness improves success from 37.3% to 64.0%. On SimplerEnv Bridge, the light agent with the playbook achieves 66.7% success, compared with 41.7% for the GR00T-only baseline, and outperforms the strong agent without a playbook. The same playbook also benefits the strong agent, which reaches 79.2% success. These results demonstrate the feasibility of harness distillation for robotics: intervention experience can be accumulated, refined through execution, and reused across agents to improve manipulation.

View source

Similar papers

Preprint Oct 2026

AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics

Agentic robotics systems combine planners with robot harnesses to perform complex manipulation tasks. Recent work improves these systems through better harnesses and external memory, but relies on powerful closed-source models as planners and does not improve the planners'own capabilities. In contrast, coding agents ha...

Xin-Xin Liu, Bing-Xin Xu, Wen-Rui Bao et al. · 2 citations · ⚡1
#natural language process... Preprint Sep 2026

Agent as Policy for Robotic Manipulation

This work demonstrates that a general-purpose agent can directly drive a physical robot throughout task execution without any task-specific or environment-specific training and introduces Agent as Policy (AGP), which places task planning and execution under the agent's control.

Mengzhao Jia, Yang Lin, Xi-Xin Zhang et al. · 23 citations · ⚡1
Sep 2026

RoboHarn-Evo: Evolving Hierarchical Physical Knowledge for Self-Improving Robotic Manipulation

RoboHarn-Evo is introduced, a dual-loop harness that evolves Hierarchical Physical Knowledge (HPK) from physical experience that demonstrates that physical interaction can be accumulated into reusable knowledge for improving subsequent manipulation.

Shi-Feng Bao, Fan-Ding Huang, Yi-Han Lin et al. · 4 citations
Preprint Oct 2026

RobotUse: Allocating Computation, Context, and Decisions

Robot agents must connect their intended actions to observed outcomes while retaining the context needed to revise their choices over repeated attempts. Existing interfaces often leave these choices inside predefined tools or require agents to manage detailed execution code and its growing history. We introduce RobotUs...

Junhoo Lee, In-Chang Baek, Seungyeon Kim et al. · 0 citations
Preprint Oct 2026

TinyAstral: Transferring Self-Evolving Harnesses to Tiny Generalist Manipulation Agents

Robot agents built on vision-language models depend on both the planner and its harness: the tools, feedback, memory, and recovery logic around it. Can improvements to a large teacher's harness be carried into a compact agent with lower inference compute cost? We present TinyAstral, a two-stage approach. We first freez...

Ke-Hong Gong, Xiao-Xu Zheng, Guan-Ling Hou et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.