Skip to content
Book Open access

Motion-Phase Collaborative Learning for Dynamic Facial Expression Recognition

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 17 references

Abstract

Dynamic Facial Expression Recognition (DFER) in the wild remains challenging due to subtle motion cues, severe temporal redundancy, temporal misalignment, and long-tailed class distributions. While masked video modeling has shown strong potential for video representation learning, its conventional pixel-level reconstruction objective is suboptimal for DFER, as it emphasizes low-level appearance statistics rather than motion-sensitive semantics. To address this, we propose Motion-Phase Collaborative Learning (MPCL), a unified two-stage framework for DFER. In the pre-training stage, we introduce Motion-Aware Dual-Region Masking (MDRM), which decomposes motion-augmented videos into facial and trajectory regions and applies region-specific masking via Temporal-Consistent Tube Masking and Trajectory-Guided Sparse Masking to enhance motion-sensitive representation learning. Building on this, we further propose CLIP-Guided Semantic Reconstruction (CGSR), which replaces pixel reconstruction with high-level semantic feature reconstruction under a frozen CLIP teacher, thereby providing semantic targets for motion-sensitive representation learning. In the fine-tuning stage, we develop Phase-Aware Progressive Fine-Tuning (PAPF), which progressively models spatiotemporal dependencies and incorporates a phase-aware supervised contrastive objective to mitigate temporal misalignment and improve robustness under long-tailed distributions. Extensive experiments on FERV39k, MAFW, and DFEW demonstrate that MPCL achieves state-of-the-art performance, with consistent improvements in both weighted and unweighted average recall.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.