Deep Multi-Task Locomotion-Invariant Gesture Recognition With FMCW Radar

Abstract

Radar-based gesture recognition (GR) remains sensitive to user motion and environmental variability. Most existing systems are designed for stationary users in controlled settings and often lose robustness during natural movement. To address this limitation, we introduce the Locomotion and Gesturing Invariant Dataset (LAGID), a large-scale FMCW radar dataset that includes hand gestures from stationary and walking users, nongesture body motions, diverse participants, and multiple environments. Building on LAGID, we propose a locomotion-aware multi-task learning framework for radar-based hand GR (RBHGR). The framework jointly learns gesture and locomotion cues, using locomotion classification as an auxiliary task to separate whole-body motion from fine-grained hand dynamics. A supervised contrastive (SC) regularization term further improves feature compactness and class separability under motion variability. Experiments show that the proposed method achieves 97.1% recognition accuracy and consistently outperforms the evaluated state-of-the-art baselines in both stationary and walking scenarios. These results indicate that the proposed method provides a practical, reliable, and privacy-preserving radar-based interface for human–machine interaction.