Skip to content
Preprint

AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learning

Aug 2026 · 1 citation · 76 references
Computer Science

TL;DR

AdvDex is presented, a unified Vision-Language-Action framework for learning dexterous manipulation from human and robot demonstrations and the Joint-Aligned Action Space (JAAS) is proposed, a canonical action representation comprising an $\mathrm{SE}(3)$ wrist pose and 15 finger joints, thereby functionally aligning human hands, dexterous robot hands, and parallel grippers.

Abstract

Dexterous manipulation is a fundamental capability for embodied intelligence, but scaling it remains difficult because robot demonstrations are expensive to collect and action spaces vary across embodiments. Policies trained on heterogeneous data can also entangle task-relevant visual cues with embodiment-specific appearance, limiting cross-embodiment generalization. We present AdvDex, a unified Vision-Language-Action framework for learning dexterous manipulation from human and robot demonstrations. First, we introduce OmniShare, a large-scale multimodal dataset of human manipulation demonstrations that provides high-quality kinematic supervision and tactile measurements while reducing reliance on robot teleoperation. Second, we propose the Joint-Aligned Action Space (JAAS), a canonical action representation comprising an $\mathrm{SE}(3)$ wrist pose and 15 finger joints, thereby functionally aligning human hands, dexterous robot hands, and parallel grippers. Finally, we use domain-adversarial learning to reduce embodiment-specific information in the learned visual representation. Experiments on hand-action prediction and real-world dexterous manipulation show consistent improvements over baselines, effective zero-shot human-to-robot skill transfer, generalization to unseen objects and environments, and data-efficient few-shot adaptation.

View source

Similar papers

Sep 2026

UniDex-ViTac: Learning Unified Visuo-Tactile Dexterous Manipulation Policy from Human Video Data

Human videos provide demonstrations of dexterous manipulation but lack robot-executable actions and tactile measurements. We present UniDex-ViTac, a framework that uses human-video-guided simulation to generate robot demonstrations paired with fingertip contact observations for training a deployable visuo-tactile polic...

Hyesung Lee, Si-Hwan Heo, Sungwook Yang · 0 citations
Preprint Sep 2026

DexWeave: Learning Dexterous Humanoid Loco-Manipulation from Human Demonstrations

Learning dexterous humanoid loco-manipulation from human demonstrations requires transferring not only human motion, but also the coordinated interaction structure underlying the demonstrated behavior. This is challenging because embodiment differences distort the coupling among body motion, wrist placement, finger art...

Nai-Chuan Sun, Hao Shen, Yi-Zhang Zhang et al. · 0 citations
Sep 2026

FoLD: Force-Informed Learning for Dexterous Articulated Object Manipulation

Transferring human demonstrations to dexterous robots remains challenging because differences in hand morphology and contact dynamics often cause retargeted motions to fail at producing the intended object behavior. We present \textbf{FoLD}, a framework for learning dexterous manipulation of articulated objects through...

Hao-Wei Shen, Ti-Ngai Li, Yu-Meng Liu et al. · 0 citations
Sep 2026

EgoWild2Dex: Learning Dexterous Robotic Manipulation from In-the-Wild Human Experience

EgoWild2Dex, which transfers in-the-wild ego-human experience to dual-arm robots with dexterous hands by jointly aligning unstable egocentric views and human motions with robot observations and actions, is introduced and GeoFormer, a differentiable geometric transformer that warps noisy human observations toward robot...

Kun-Li Channing Lin, Xu-Tao Wen, Jing Lin et al. · 0 citations
Sep 2026

SkelWAM: A Skeleton-Guided World-Action Model for Zero-Shot Cross-Embodiment Manipulation

This work presents SkelWAM, a skeleton-guided world-action model that couples perception and control through one explicit geometric representation for single-source cross-embodiment manipulation, and introduces LIBERO-Cross10, a source-only cross-embodiment transfer benchmark covering ten tasks and ten target embodimen...

Pengbo Niu, Yu-Jia Xie, Rui Peng et al. · 0 citations
Preprint Sep 2026

DexTouch-WM: Learning Action-Conditioned Tactile World Models from Human Touch for Dexterous Robot Manipulation

Learning predictive models of contact-rich dexterous manipulation requires dense tactile interaction, but such data are costly to scale on real robots and remain tied to embodiment-specific sensors. We introduce DexTouch-WM, an action-conditioned world model that learns from scalable human touch to jointly predict futu...

Yan-Jiao Qin, Yue Chen, Wen-Wei Lin et al. · 3 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.