A Multimodal Framework for Enhanced Deepfake Detection Using rPPG and Iris Features
Abstract
Deepfakes generated by advanced deep learning models pose serious threats to privacy, public trust, and security. Conventional detection methods that rely primarily on visual artifacts are becoming less effective as synthetic media becomes increasingly realistic. This study proposes a multimodal deepfake detection framework that combines physiological and biometric cues using remote photoplethysmography (rPPG) signals and iris pattern features. The proposed system extracts facial rPPG signals from the green color channel and derives frequency-domain features through spectral analysis, while iris regions are processed using a multi-scale Gabor filter bank for texture descriptor extraction. A CNN-BiLSTM architecture models the temporal rPPG signals, while an MLP processes the FFT-based frequency-domain features. The Transformer-based encoder processes the sequential features of the iris pattern. Feature representations are integrated through both concatenation and attention-based fusion mechanisms. Experiments were conducted on the FaceForensics++ dataset using a five-fold crossvalidation protocol with multiple evaluation metrics, including AUC, accuracy, precision, recall, and f1-score. The proposed multimodal framework achieved the best performance without attention, with an average AUC of 0.9676 and an accuracy of 91.86%, compared with 0.9606 and 91.40%, respectively, for the attention-based model. The attention mechanism did not provide a statistically significant improvement in either AUC or accuracy. These results suggest that the primary performance gain comes from the complementary integration of rPPG and iris features rather than the addition of the attention mechanism.