InfiniHand is an end-to-end streaming feed-forward framework that jointly estimates MANO parameters, camera trajectories, and hand locations directly from uncalibrated egocentric video, achieving a 21% reduction in ARCTIC PA-p compared to ViDiHand while substantially mitigating world-space drift.
Abstract
World-space hand motion estimation from egocentric video requires recovering 3D articulated hand geometry while tracking camera egomotion. Existing approaches heavily rely on cascading independent hand pose estimators and SLAM systems, resulting in error accumulation, complex pipelines, and severe computational overhead. To address these limitations, we present InfiniHand, an end-to-end streaming feed-forward framework that jointly estimates MANO parameters, camera trajectories, and hand locations directly from uncalibrated egocentric video. InfiniHand integrates persistent spatiotemporal memory with hand-centered visual features, explicitly coupling camera motion with local hand geometry within a unified architecture. We train InfiniHand in two progressive stages by first learning robust camera-space hand priors and then extending to streaming world-space reconstruction. To support this process, we aggregate a pretraining corpus of approximately 5,000 hours of egocentric data across multiple public datasets. Extensive evaluations demonstrate that InfiniHand outperforms state-of-the-art baselines on in-domain benchmarks, achieving a 21.4% reduction in ARCTIC PA-p compared to ViDiHand while substantially mitigating world-space drift. Furthermore, InfiniHand generalizes robustly to in-the-wild videos and operates at 11.19 FPS, delivering more than twice the throughput of HaWoR.
This work introduces MINT (Minting IN-the-Wild Trajectories), a foundation model for world-space hand motion reconstruction from ego-centric RGB video and develops an open-source labeling EGOPIPELINE that converts large collections of public egocentric videos into structured camera-and-hand trajectory supervision.
Zi-Jie Zhu, Wei-Ren Cai, Yi-Zhou Wang et al.· 3 citations
EgoFound3R, a unified end-to-end model that estimates world-space hand geometry in a metric scale shared with the scene, and predicts point-wise interaction attributes, including visibility, contact, and distance, is introduced.
Hong-Ming Fu, Jing-Cheng Shi, Wen-Jia Wang et al.· 0 citations
ACE-Ego-Hand is introduced, an offline clip-level framework that extracts features via a Deterministic Clean-Latent Encoder and decodes them with a Bidirectional Spatiotemporal Decoder, offering a scalable path from everyday human video to robot manipulation data.
Yu-Fei Liu, Xixi Wang, Hao Li et al.· 4 citations· ⚡2
Experiments show that RESELF outperforms state-of-the-art methods designed for the individual tasks across depth estimation, camera tracking, and full-body motion estimation.
Kai Guan, Minchao Jiang, Ruichen WangLi et al.· 0 citations
DiffWAM is presented, a geometry-conditioned navigation world-action model that directly transforms multi-level predictive features into continuous camera trajectories, providing a direct alternative to generate-then-reconstruct navigation pipelines.
Morui Zhu, Yu-Ze Wu, Xi-Jie Huang et al.· 1 citation
MEgoVista is offered as a measured route from egocentric video to metric hand supervision, one that widens where such labels can be gathered, one that widens where such labels can be gathered.
Jian-Gong Xiao, Zhi-Hao Zhang, Yi-Fei Dong et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.