A multimodal prosthetic hand control method integrating sEMG and visual recognition

Abstract

Surface electromyography (sEMG) signals have been widely used in prosthetic hand control research because they can reflect muscle activity. However, control methods based on residual-limb sEMG are susceptible to factors such as muscle atrophy and electrode displacement, leading to limitations in gesture adaptability and intention recognition. In addition, existing classification models often suffer from relatively high latency. To address these issues, this paper proposes a multimodal prosthetic hand control method that integrates sEMG signals from the shoulder on the intact side with visual recognition. In this method, shoulder movements on the intact side are used as low-dimensional intention-triggering signals, while a vision module is employed to identify the categories of target objects. Combined with a biomimetic grasping trajectory library constructed based on Vicon motion capture, the proposed method enables natural grasp control for daily objects. Meanwhile, a lightweight Adaptive-Pruning LSTM (AP-LSTM) model is designed. Through grasping experiments conducted with 10 participants, the system achieved rapid execution in grasping tasks involving six types of daily objects, with an average grasping time of 3.1 s and a task success rate of 92.5%. The experimental results demonstrate that the proposed method exhibits potential feasibility in sEMG-based intention recognition and natural grasp control, while the subjective assessment indicates that the low-dimensional trigger strategy is acceptable for most participants in short-term use.