Fractal-geometric prompt modulation and multi-component pseudo-label reconstruction for facial expression recognition in human-computer interaction

Abstract

Facial expression recognition in the wild remains challenging due to large intra-class variation, identity interference, and the scarcity of high-quality annotations. To address these issues, this paper proposes a novel facial expression recognition framework based on Fractal-Geometric Prompt Modulation (FGPM) and multi-component pseudo-label reconstruction (MCPLR). The proposed framework adopts a dual-branch architecture, where an iResNet18 backbone extracts facial texture representations and a lightweight MobileFaceNet captures geometric semantic information from facial landmarks. A Fractal-Geometric Prompt Modulation module is then introduced to transform landmark-derived geometric cues into learnable prompts for adaptive modulation of texture features. To further exploit unlabeled samples, a semi-supervised knowledge transfer strategy based on multi-component pseudo-label reconstruction is proposed, which transfers target confidence, non-target class distribution, and decision statistics from an auxiliary model to the target model. In addition, an adaptive fractional-order optimizer is employed to improve training efficiency and convergence behavior. Extensive experiments on multiple benchmark datasets, including RAF-DB, SFEW, and RaFD, demonstrate the effectiveness of our proposed method. Specifically, it achieved an accuracy of 92.3% and a WF1 score of 0.922 on RAF-DB; an accuracy of 61.4% and a WF1 score of 0.579 on SFEW; and on the RaFD-F, T, and P subsets, it achieved accuracy and WF1 scores of 99.8% and 0.998, 99.3% and 0.993, and 98.2% and 0.982, respectively. The results indicate that our proposed method achieves higher recognition accuracy than many mainstream approaches while maintaining low model complexity.