This work introduces RoboSkill, a framework that connects skill acquisition and reuse through an Explore, Execute, Evolve loop, and complements vision with tactile feedback to reduce uncertainty during physical interaction and augment textual guidance with reusable code to reduce reasoning overhead during skill reuse.
Abstract
Vision-language-action and world-action models have demonstrated impressive capabilities in robotics, yet generalization to unseen tasks remains challenging. More recently, general-purpose multimodal agents have shown great potential for zero-shot robotic task solving. However, they often incur high execution costs by reasoning and exploring the physical world from scratch. To reduce these costs, we introduce RoboSkill, a framework that connects skill acquisition and reuse through an Explore, Execute, Evolve loop. Within this loop, the agent explores to gather task-relevant information, executes tasks while adapting to feedback, and evolves its skill library based on execution records. It then reuses these skills to guide exploration and execution in the next cycle, closing the loop. To improve loop efficiency, we complement vision with tactile feedback to reduce uncertainty during physical interaction. We further augment textual guidance with reusable code to reduce reasoning overhead during skill reuse. On LIBERO-10, RoboSkill improves first-episode success rates by 12.5--25.0 percentage points and reduces average runtime by 7.6--72.4% across four agents. On real robots, it improves success rates by 8.3 percentage points and reduces average runtime for successful trials by at least 14.4%.
Building reliable robot capabilities across diverse tasks requires substantial human effort to develop and maintain skills, design rewards, and integrate perception with control. We present Reconstruct, Practice, Go Real (RPG), a framework for autonomous improvement of robot execution systems without updating model wei...
Yen-Jen Wang, Hao-Zhe Jiang, Shu-Ying Deng et al.· 11 citations
Large-scale demonstrations have driven unprecedented progress in robot learning, yet collecting robot data through teleoperation is expensive and difficult to scale to diverse environments and long-horizon tasks. Simulation offers a scalable alternative, but existing data-generation pipelines often rely on open-loop co...
He Zhu, Lu-Sen Zhao, Kwan-Man Cheng et al.· arXiv.org· 1 citation
RoboHarn-Evo is introduced, a dual-loop harness that evolves Hierarchical Physical Knowledge (HPK) from physical experience that demonstrates that physical interaction can be accumulated into reusable knowledge for improving subsequent manipulation.
Shi-Feng Bao, Fan-Ding Huang, Yi-Han Lin et al.· arXiv.org· 4 citations
ASENA, an embodied agent system that enables general-purpose coding agents to program humanoid navigation, manipulation, and interaction, is presented and ASENA-VLN, a 4B monocular navigation policy that serves as an optional tool within this programmable system, is introduced.
An-Chieh Cheng, Isabella Liu, Edmund Bu et al.· arXiv.org· 2 citations
A generalist robot should not only perform diverse tasks but also improve through experience, turning what it learns during execution into capabilities that later tasks can reuse. Robot agents that act through code can already repair programs from execution feedback, yet it remains a central challenge to organize this...
Zi-Mo Wen, Yi-Jin Chen, Yuxuan Cao et al.· 10 citations· ⚡2
The proposed RoboBridge framework provides a route from one-shot policy deployment to continual procedural learning across environments, and preserves reusable task structure while allowing environment-dependent operations to be selectively revised through real-world execution feedback.
Chen-Xi Li, Zhang-Rui Zhao, Rui Li et al.· 3 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduJul 30, 2026
From feet to fingertips — we are teaching robots intelligent whole-body control, fine dexterity, and teamwork to complete a broad range of complex tasks.