MoDex is proposed, a data-efficient framework that provides the first unified solution for universal multi-object dexterous grasping across diverse spatial layouts by formulating the task as a single-step decision problem of learning residual corrections to a reference trajectory, and introducing a spatially adaptive dual-strategy.
Abstract
Multi-object grasping is a fundamental yet challenging problem, constrained by data collection costs, varying spatial layouts, and the need for broad generalization. To address these challenges, we propose MoDex, a data-efficient framework that provides the first unified solution for universal multi-object dexterous grasping across diverse spatial layouts. MoDex accomplishes this by formulating the task as a single-step decision problem of learning residual corrections to a reference trajectory, and introducing a spatially adaptive dual-strategy that performs single-shot grasping for proximal objects and sequential interaction for separated objects. Extensive evaluations show that MoDex significantly outperforms baselines in proximal configurations, achieving 61.24% success on unseen objects, and achieves high success rates in separated layouts. Further experiments demonstrate its generalization across robot embodiments, scalability and zero-shot sim-to-real deployment through a vision-based policy trained solely on simulation data. Code and pretrained models are available at https://github.com/Aurora-Xiong/MoDex.
Dexterous grasping is the foundational primitive in embodied AI, demanding massive data to train robust models. As real-world data collection is expensive, simulation has become the mainstream paradigm. Yet, while cluttered scenes best reflect real-world applications, learning to grasp within them is bottlenecked by a...
N. Fang, Zhong-Jin Luo, Yu-Xin Mo et al.· 0 citations
This work presents ArtManip, the first category-level articulated in-hand manipulation method that generalizes across object instances and diverse initial grasps and proposes a robust two-stage training strategy that incorporates articulation physics randomization, reward curriculum, and latent representation distillat...
Yang Yang, Teng-Yu Liu, Pu-Hao Li et al.· arXiv.org· 1 citation
Achieving stable and diverse dexterous grasping of deformable objects remains a fundamental challenge in robotics, due to high-dimensional action spaces and complex material properties. In this paper, we propose DeformableGrasp, a multimodal perception-guided reinforcement learning framework that enables adaptive grasp...
Ke-Yu Wang, Bing-Cong Lu, Zheng-Xue Cheng et al.· Proceedings of the 2nd Inter...· 0 citations
This work construct a digital twin of the environment, query a large foundation model to propose grasps that align with the object's affordances and task description, and then optimize the proposals to ensure robustness and plausibility before executing the result on the real robot.
Daniel J. Evans, Yin-Long Dai, Simon Stepputtis et al.· arXiv.org· 0 citations
DeviGrasp-Bench is introduced, a benchmark for mobile grasping under degraded vision that incorporates controlled visual degradations, seen and unseen objects, multiple difficulty levels, and complex terrains, and evaluates task success, execution efficiency, and action smoothness, and DeViGrasp-Net is proposed, a teac...
Liang Zhou, Jia-Ming Su, Yang Wei et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.