EgoAlign, a data-construction framework that converts these demonstrations into action and state supervision compatible with a general-purpose, continuous whole-body controller, without collecting physical-robot demonstrations, is presented.
Abstract
Egocentric human demonstrations offer an accessible source of task experience, but differences in body scale and controller response, together with missing robot states, limit their value as humanoid training supervision. We present EgoAlign, a data-construction framework that converts these demonstrations into action and state supervision compatible with a general-purpose, continuous whole-body controller, without collecting physical-robot demonstrations. Using the target-robot model and simulator, EgoAlign guides demonstration collection through execution feedback. It preserves locomotion references for visually guided periodic stepping while adapting upper-body interaction geometry through scale alignment and controller-in-the-loop refinement. A final causal replay reconstructs the corresponding robot states and motion-token labels for training with the human observations. We assess the resulting supervision by fine-tuning a vision--language--action model solely on adapted human demonstrations and deploying it zero-shot on a physical humanoid. The resulting policies perform long-range object relocation, navigation to unseen goal positions, and independently evaluated foot interaction. Refinement improves simulated hand alignment and physical pickup success over kinematic alignment alone, while human collection reduces on-site acquisition time relative to teleoperation. https://lambdahumanoid.github.io/EgoAlign/
EgoHumanoid-V2 is presented, the first egocentric human-to-humanoid skill transfer framework for coordinated whole-body loco-manipulation, which improves end-effector pose accuracy while preserving whole-body coordination and uses robot-arm rendering and training-time image augmentation to reduce the visual embodiment...
Jin Chen, Yi-Ming Jiang, Chong-Yang Xu et al.· arXiv.org· 2 citations
Experimental results demonstrate that a single policy enables a Unitree G1 humanoid to complete the full task using only onboard depth sensing and proprioception, while generalizing robustly across task variations and transferring effectively from simulation to reality.
Ze-Jie Tian, Rui-Bing Hou, Bin Ma et al.· arXiv.org· 0 citations
Humanoid whole-body manipulation has advanced rapidly, enabling policies to coordinate locomotion, posture, bimanual interaction, and dexterous hand movements. Meanwhile, egocentric human videos provide diverse examples of everyday interactions across objects and scenes, offering scalable supervision without robot oper...
Chong-Yang Xu, Zhao Wu, Jin Chen et al.· 0 citations
Humanoid robots still struggle to plan contact-rich whole-body manipulation from egocentric RGB and proprioception. Workhorse learns such manipulation from robot-free human demonstrations. A visual planner predicts five-link targets: the poses of the torso, both wrists, and both feet. A reinforcement-learning whole-bod...
Song-Bo Hu, Qia-Yuan Liao, Yu-Feng Chi et al.· 1 citation
Skel-WAM is introduced, a world action model that bridges differences between human and robot motion through a unified hand-skeleton motion interface, and demonstrates that a shared skeletal interface enables joint learning across human and robot data and expands robot task coverage through complementary human demonstr...
Ze-Tao Cai, Ya-Ping Li, Yi-Qun Wang et al.· arXiv.org· 1 citation
Egocentric human data provides a scalable source of experience for robot learning, but varies substantially in human-robot alignment, behavioral coverage, and available supervision. Existing work shows favorable scaling with increasing human data, but it remains unclear which data properties drive downstream robot gain...
Zhihao Sun, Liu Liu, Xin-Jiang Wang et al.· arXiv.org· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.