Fine-grained multi-level gesture recognition based on a stretchable multichannel ultrasonic device
TL;DR
A stretchable multichannel ultrasonic device comprising four functional sites and sixteen piezoelectric modules yields high classification accuracy, improved class-wise recognition balance, more effective learning from multi-subject data and enhanced calibration-assisted adaptation to shifted device positions and new users, providing a reliable strategy for low-burden, fine-grained multi-level gesture command control.
Abstract
Discrete gesture recognition provides a direct output form for command-based human-machine interaction, while fine-grained multi-level recognition can expand command capacity by mapping subtle graded finger movements to distinct commands or different levels of the same command, thereby reducing the need for large or repetitive hand gestures. However, reliably distinguishing fine-grained multi-level gestures remains challenging. Here, we present a stretchable multichannel ultrasonic device comprising four functional sites and sixteen piezoelectric modules. Its fan-shaped substrate stretches up to 30%, enabling conformal forearm attachment and alignment with target muscle regions. Experimental characterization demonstrated sub-millimeter spatial resolution and excellent signal quality. Integrated with a one-dimensional convolutional neural network, the system achieved up to 98.75% accuracy in recognizing metacarpophalangeal joint-angle changes below 5°. The multichannel configuration yields high classification accuracy, improved class-wise recognition balance, more effective learning from multi-subject data and enhanced calibration-assisted adaptation to shifted device positions and new users, providing a reliable strategy for low-burden, fine-grained multi-level gesture command control.