Cone-Constrained Human-in-the-Loop Shared Control with 3D Diffusion Policy for Dexterous Tool Manipulation

Abstract

Dexterous hands for intelligent manufacturing are being extensively studied with the development of humanoid robot technology. Nevertheless, fine-grained human-robot collaborative dexterous hand tool operations remain challenging because they require precise control, while teleoperation imposes a high burden on the operator during precise approach and grasping. Even worse, learning-based methods may fail when real-world conditions deviate from the training distribution. To address these challenges, this study introduces cone-constrained gated human-in-the-loop shared control for dexterous hand tool manipulation based on a 3D diffusion policy (DP3), ConeGate-DP3. First, the study designed a bilateral teleoperation exoskeleton with wrist force feedback for shared control in the system. This teleoperation exoskeleton not only enables human experts to complete global approach and anomaly handling but also conveys the robot's intention to the expert through a dedicated force feedback device. Second, the DP3 policy in the system takes over during local phases once both the operator and the system confirm takeover feasibility to accomplish critical fine-grained tool operations, thereby balancing robustness and operational consistency. Third, this study proposes a geometry-based method to assess takeover feasibility from the relative configuration between the endeffector and the target tool. Validation in real-world humanrobot collaborative dexterous hand tool handling demonstrates the effectiveness of the proposed system, which combines the advantages of human global decision-making, tactile guidance, and the fine-grained local control of diffusion strategies, making a significant contribution to the field of human-in-the-loop shared control and dexterous tool manipulation.