Experimental results demonstrate that a single policy enables a Unitree G1 humanoid to complete the full task using only onboard depth sensing and proprioception, while generalizing robustly across task variations and transferring effectively from simulation to reality.
Abstract
Humanoid loco-manipulation requires adaptive whole-body coordination to seamlessly integrate locomotion and physical interaction. Despite recent advances, learning autonomous loco-manipulation remains challenging due to the scarcity of diverse, physically executable robot-object interaction data and the difficulty of learning unified whole-body control directly from onboard observations. We present ViLoMan, a scalable framework for autonomous humanoid loco-manipulation. ViLoMan first transforms partial kinematic demonstrations of human-object interactions into complete, physically executable robot trajectories. It then leverages these trajectories within a teacher-student distillation framework to learn a unified policy that maps egocentric depth observations and proprioceptive measurements directly to joint-level whole-body actions. During deployment, the policy requires neither reference motions nor intermediate commands. We evaluate ViLoMan on door-closing tasks across diverse door configurations and robot initial conditions in both simulation and the real world. Experimental results demonstrate that a single policy enables a Unitree G1 humanoid to complete the full task using only onboard depth sensing and proprioception, while generalizing robustly across task variations and transferring effectively from simulation to reality. Project page: viloman-anonymous.pages.dev.
Weave is presented, a unified framework for learning whole-body dexterous humanoid-object interaction from captured human demonstrations that converts captured human-object interactions into executable robot-object references through contact-aware retargeting and approach-motion completion.
Liu Cao, Xing-Ze Wu, Jing-Zhi Cui et al.· 2 citations
EgoHumanoid-V2 is presented, the first egocentric human-to-humanoid skill transfer framework for coordinated whole-body loco-manipulation, which improves end-effector pose accuracy while preserving whole-body coordination and uses robot-arm rendering and training-time image augmentation to reduce the visual embodiment...
Jin Chen, Yi-Ming Jiang, Chong-Yang Xu et al.· arXiv.org· 2 citations
Humanoid whole-body manipulation has advanced rapidly, enabling policies to coordinate locomotion, posture, bimanual interaction, and dexterous hand movements. Meanwhile, egocentric human videos provide diverse examples of everyday interactions across objects and scenes, offering scalable supervision without robot oper...
Chong-Yang Xu, Zhao Wu, Jin Chen et al.· 0 citations
EgoAlign, a data-construction framework that converts these demonstrations into action and state supervision compatible with a general-purpose, continuous whole-body controller, without collecting physical-robot demonstrations, is presented.
Yi-Ming Jiang, Jin Chen, Chong-Yang Xu et al.· arXiv.org· 0 citations
Learning dexterous humanoid loco-manipulation from human demonstrations requires transferring not only human motion, but also the coordinated interaction structure underlying the demonstrated behavior. This is challenging because embodiment differences distort the coupling among body motion, wrist placement, finger art...
Nai-Chuan Sun, Hao Shen, Yi-Zhang Zhang et al.· 0 citations
This work presents RobotMover, a complete learning-based system for large-object manipulation that leverages human–object interaction demonstrations to train robot control policies and achieves strong performance in terms of capability, robustness, and controllability, outperforming both learned and teleoperation basel...
Tian-Yu Li, Joanne Truong, Tsung-Yen Yang et al.· IEEE Transactions on robotic...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.