Skip to content

ViLoMan: Learning Visual-Proprioceptive Whole-Body Loco-Manipulation Skills for Humanoid Robots

Sep 2026 · arXiv.org · Vol abs/2609.19340 · 0 citations · 33 references
Computer Science

TL;DR

Experimental results demonstrate that a single policy enables a Unitree G1 humanoid to complete the full task using only onboard depth sensing and proprioception, while generalizing robustly across task variations and transferring effectively from simulation to reality.

Abstract

Humanoid loco-manipulation requires adaptive whole-body coordination to seamlessly integrate locomotion and physical interaction. Despite recent advances, learning autonomous loco-manipulation remains challenging due to the scarcity of diverse, physically executable robot-object interaction data and the difficulty of learning unified whole-body control directly from onboard observations. We present ViLoMan, a scalable framework for autonomous humanoid loco-manipulation. ViLoMan first transforms partial kinematic demonstrations of human-object interactions into complete, physically executable robot trajectories. It then leverages these trajectories within a teacher-student distillation framework to learn a unified policy that maps egocentric depth observations and proprioceptive measurements directly to joint-level whole-body actions. During deployment, the policy requires neither reference motions nor intermediate commands. We evaluate ViLoMan on door-closing tasks across diverse door configurations and robot initial conditions in both simulation and the real world. Experimental results demonstrate that a single policy enables a Unitree G1 humanoid to complete the full task using only onboard depth sensing and proprioception, while generalizing robustly across task variations and transferring effectively from simulation to reality. Project page: viloman-anonymous.pages.dev.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Weave: Learning Whole-Body Dexterous Loco-Manipulation from Human-Object Interactions

Weave is presented, a unified framework for learning whole-body dexterous humanoid-object interaction from captured human demonstrations that converts captured human-object interactions into executable robot-object references through contact-aware retargeting and approach-motion completion.

Liu Cao, Xing-Ze Wu, Jing-Zhi Cui et al. · 2 citations

EgoHumanoid-V2: Human-to-Humanoid Transfer of Coordinated Whole-Body Skills for Loco-Manipulation

EgoHumanoid-V2 is presented, the first egocentric human-to-humanoid skill transfer framework for coordinated whole-body loco-manipulation, which improves end-effector pose accuracy while preserving whole-body coordination and uses robot-arm rendering and training-time image augmentation to reduce the visual embodiment...

Jin Chen, Yi-Ming Jiang, Chong-Yang Xu et al. · 2 citations
Preprint Sep 2026

Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining

Humanoid whole-body manipulation has advanced rapidly, enabling policies to coordinate locomotion, posture, bimanual interaction, and dexterous hand movements. Meanwhile, egocentric human videos provide diverse examples of everyday interactions across objects and scenes, offering scalable supervision without robot oper...

Chong-Yang Xu, Zhao Wu, Jin Chen et al. · 0 citations
Sep 2026

EgoAlign: Bridging the Human-Humanoid Gap for Long-Range Loco-Manipulation

EgoAlign, a data-construction framework that converts these demonstrations into action and state supervision compatible with a general-purpose, continuous whole-body controller, without collecting physical-robot demonstrations, is presented.

Yi-Ming Jiang, Jin Chen, Chong-Yang Xu et al. · 0 citations
Preprint Sep 2026

DexWeave: Learning Dexterous Humanoid Loco-Manipulation from Human Demonstrations

Learning dexterous humanoid loco-manipulation from human demonstrations requires transferring not only human motion, but also the coordinated interaction structure underlying the demonstrated behavior. This is challenging because embodiment differences distort the coupling among body motion, wrist placement, finger art...

Nai-Chuan Sun, Hao Shen, Yi-Zhang Zhang et al. · 0 citations
Open access Feb 2025

RobotMover: Learning to Move Large Objects From Human Demonstrations

This work presents RobotMover, a complete learning-based system for large-object manipulation that leverages human–object interaction demonstrations to train robot control policies and achieves strong performance in terms of capability, robustness, and controllability, outperforming both learned and teleoperation basel...

Tian-Yu Li, Joanne Truong, Tsung-Yen Yang et al. · 2 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.