CEE-NeRF: Dynamic Neural Radiance Fields With Contrastive Enhanced Expression Representation for 4-D Face Reconstruction

Abstract

The reconstruction of photorealistic 4-D faces from monocular video holds significant importance for applications such as virtual reality and digital human-robot interaction. However, achieving reconstructions with structurally reasonable facial geometry and natural expressions remains challenging. Existing approaches typically combine dynamic neural radiance fields (NeRFs) with low-dimensional parametric models, such as 3-D morphable model (3DMM), for facial reconstruction and expression control. Nevertheless, 3DMM-based expression features often lack the ability to preserve high-frequency details and subtle individual variations. To address this issue, we propose contrastive expression enhancement (CEE)-NeRF, a novel framework that integrates geometry-aware 3DMM features with high-level semantic features through contrastive learning for complementary feature fusion. This design enhances the representation of fine-grained facial details while mitigating expression artifacts. In addition, unlike previous studies, we propose reformulating monocular 4-D face reconstruction as a multimodal learning problem and replacing the conventional multilayer perceptron used in NeRF with a mixture-of-experts Transformer, which enables modality-specific expert processing for accurate density and color estimation. This multimodal framework is also highly extensible and can be easily adapted to additional modalities. Extensive experiments demonstrate that CEE-NeRF achieves high-fidelity reconstructions from challenging monocular videos, consistently outperforming state-of-the-art approaches in rendering quality.