The results demonstrate the feasibility of harness distillation for robotics: intervention experience can be accumulated, refined through execution, and reused across agents to improve manipulation.
Abstract
A central goal in robotics is to enable manipulation across changing tasks and environments. Vision-language-action (VLA) models provide broad manipulation capabilities but can struggle when execution requires diagnosing failures and adapting behavior. Strong agents can discover effective interventions through interaction with these policies. We propose Recursive Harness Distillation to accumulate this experience as reusable guidance across agents. A strong agent distills its experience into a playbook for a light agent, then recursively refines the playbook using the light agent's execution feedback. The resulting playbook enables agents to reuse accumulated intervention knowledge in new task instances without updating model parameters. In real-world manipulation, the harness improves success from 37.3% to 64.0%. On SimplerEnv Bridge, the light agent with the playbook achieves 66.7% success, compared with 41.7% for the GR00T-only baseline, and outperforms the strong agent without a playbook. The same playbook also benefits the strong agent, which reaches 79.2% success. These results demonstrate the feasibility of harness distillation for robotics: intervention experience can be accumulated, refined through execution, and reused across agents to improve manipulation.
Agentic robotics systems combine planners with robot harnesses to perform complex manipulation tasks. Recent work improves these systems through better harnesses and external memory, but relies on powerful closed-source models as planners and does not improve the planners'own capabilities. In contrast, coding agents ha...
Xin-Xin Liu, Bing-Xin Xu, Wen-Rui Bao et al.· 2 citations· ⚡1
This work demonstrates that a general-purpose agent can directly drive a physical robot throughout task execution without any task-specific or environment-specific training and introduces Agent as Policy (AGP), which places task planning and execution under the agent's control.
Mengzhao Jia, Yang Lin, Xi-Xin Zhang et al.· 23 citations· ⚡1
RoboHarn-Evo is introduced, a dual-loop harness that evolves Hierarchical Physical Knowledge (HPK) from physical experience that demonstrates that physical interaction can be accumulated into reusable knowledge for improving subsequent manipulation.
Shi-Feng Bao, Fan-Ding Huang, Yi-Han Lin et al.· arXiv.org· 4 citations
Robot agents must connect their intended actions to observed outcomes while retaining the context needed to revise their choices over repeated attempts. Existing interfaces often leave these choices inside predefined tools or require agents to manage detailed execution code and its growing history. We introduce RobotUs...
Junhoo Lee, In-Chang Baek, Seungyeon Kim et al.· 0 citations
Robot agents built on vision-language models depend on both the planner and its harness: the tools, feedback, memory, and recovery logic around it. Can improvements to a large teacher's harness be carried into a compact agent with lower inference compute cost? We present TinyAstral, a two-stage approach. We first freez...
Ke-Hong Gong, Xiao-Xu Zheng, Guan-Ling Hou et al.· 1 citation
RoboArchitect shows that harness design can be automated, offering a strong alternative to finetuning on a novel setup, and its design can be effectively automated and optimized without weight updates.
Sheng-Guang Wu, Su-Ning Huang, Cherie Ho et al.· 0 citations