A Wearable Multimodal Teleoperation Interface for In-Contact Robotic Tasks

Abstract

In-contact teleoperation tasks, such as surface polishing and fragile object manipulation, require coordinated control of motion and interaction force as well as timely adjustments from the human operator. This paper presents a wearable multimodal teleoperation framework that integrates a data-driven wearable control interface with vibrotactile biofeedback. Using a single commercial armband, the proposed interface estimates both the operator's hand motion and intended interaction force from inertial measurements and surface electromyography (sEMG) signals, enabling the generation of end-effector pose and force commands simultaneously. To assist the operator during task execution, a vibrotactile device delivers biofeedback that encodes the inferred force command, allowing the user to regulate interaction force even before physical contact occurs. The framework also supports intuitive switching between Cartesian motion control and hybrid motion-force control for different task stages. A user study involving two representative teleoperation tasks—surface polishing and fragile object pick-and-place—verifies the effectiveness of the proposed robot control framework.