Diverse Spatial-Temporal Feature Enhancement and Adaptive Fusion Network for Device-Free Gesture Recognition Under Limited Labeled Data

Abstract

Deep learning (DL) has shown significant potential in device-free gesture recognition (DFGR), enabling contactless device control and intuitive human–robot interaction. However, DL-based methods usually require a large number of labeled samples for training. Since the collection of annotations for radar signals is time–consuming and labor–intensive, it is crucial to develop algorithms that can achieve high performance with limited training samples. Despite recent advances, existing DL-based models for DFGR with few labeled samples still suffer from insufficient feature discrimination and severe overfitting, which hinders their generalization ability. To address these issues, we propose a diverse spatial-temporal feature enhancement and adaptive fusion network (DSTFE-AFN) tailored for DFGR with few labeled samples. Specifically, we devise a diverse spatial-temporal enhancement module (DSTEM) to jointly exploit and reinforce the spatial and temporal features of gestures, enabling the extraction of more discriminative representations from limited data. Furthermore, we introduce an adaptive decision fusion module (ADFM) to aggregate predictions from multiple enhanced feature streams. By leveraging the diversity among these predictions, the ADFM effectively mitigates overfitting and improves model robustness. Extensive experiments conducted on human gesture recognition datasets demonstrate the superiority of the proposed model over existing state–of–the–art models for DFGR with few labels. Note to Practitioners—This paper was motivated by the practical challenge of deploying device-free gesture recognition (DFGR) systems using millimeter-wave radar, where collecting and labeling large–scale training datasets is both labor–intensive and time–consuming. Existing deep learning approaches often struggle with “few-shot” scenarios, leading to overfitting and poor generalization when data is scarce. This paper proposes a new framework, the diverse spatial-temporal feature enhancement and adaptive fusion network (DSTFE-AFN), designed to achieve high accuracy with limited labeled samples. The proposed method works by extracting gesture features at multiple spatial scales and temporal differences, effectively enhancing the discriminative information available in small datasets. It uses an adaptive fusion mechanism to intelligently combine predictions from parallel network branches, which balances their contributions to minimize overfitting. Extensive experiments on real–world radar datasets demonstrate that this approach significantly outperforms state–of–the–art methods in accuracy and stability while remaining computationally efficient for real–time applications. While the current system is validated on specific gesture sets, future research will extend this framework to cross–domain scenarios and open–set recognition tasks where the system must handle undefined or unseen gesture types.