Skip to content
Preprint

Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection

Jul 2026 · 0 citations · 25 references
Computer Science

TL;DR

A detection framework is proposed that extracts per-video rPPG wave- forms via RhythmFormer and trains a suite of lightweight classifiers to distinguish real from synthesized physiologi- cal signals and shows that detec- tion difficulty is strongly method-dependent.

Abstract

Talking-face (TF) deepfake generation synthesizes photore- alistic facial video from a static source image and an au- dio signal, producing forgeries that current image-based detectors consistently fail to identify. Unlike face-swap ma- nipulation, TF synthesis has no underlying real video from which to inherit physiological characteristics, making re- mote photoplethysmography (rPPG) a uniquely motivated detection modality for this forgery category. We propose a detection framework that extracts per-video rPPG wave- forms via RhythmFormer and trains a suite of lightweight classifiers to distinguish real from synthesized physiologi- cal signals. Evaluated on the TF subset of Celeb-DF++ un- der a strict subject-independent protocol, where test identi- ties are completely separated from training identities, our 1D ResNet achieves an AUC of 0.806 and EER of 27.8%, placing it within 2.4 points of the best published general- purpose detector (Effort, ICML 2025) while operating ex- clusively on the physiological channel. We document a con- trolled reproduction study of DeepFakesON-Phys, the rep- resentative prior rPPG detector, demonstrating degrada- tion from AUC 0.999 on legacy face-swap data to 0.622 on the TF subset of Celeb-DF++. We further show that detec- tion difficulty is strongly method-dependent: AUC ranges from 0.985 (Real3DPortrait) to 0.690 (IP-LAP) across the seven TF generators, with the ranking remaining perfectly stable across all evaluation protocols. This spread reflects an interpretable physiological property of each generator rather than evaluation noise, and constitutes the primary theoretical contribution of the work.

View source

Similar papers

Open access Jul 2026

ForensicNet: Lightweight Attention-Enhanced MobileNetV2 for Automated Face Identification

In forensic environments, automated identification of perpetrators is difficult due to pose changes, changes in light, occlusion, and lack of labeled data. This paper presents ForensicNet, a lightweight deep learning framework for forensic face recognition that enhances attention. The suggested model combines the MobileNetV2 backbone with Convolutional Block Attention Modules (CBAM) to improve the learning of discriminative features while maintaining computational speed. A two-phase transfer learning strategy with adaptive layer unfreezing is used to improve domain adaptation and reduce overfitting. This study used publicly available datasets such as LFW and SCFace, with 15,000 facial images spanning 68 identity classes. The proposed model outperforms baseline architectures such as AlexNet, ResNet-50, and MobileNetV2, with an accuracy of 92.4%, a precision of 90.8%, and a recall of 89.5%. Additionally, the framework requires only 2.1 GFLOPs per inference, and hence can be used in real-time forensic surveillance applications.

S. N. J., L. B. T. · 0 citations
Open access Jul 2026

BlinkNet: GRU-Based Temporal Analysis for Deepfake Video Detection

The increasing realism of synthetic facial videos has reduced the reliability of detectors that depend only on visible artifacts in isolated frames. This paper presents BlinkNet, an explainable deepfake-detection framework that examines the spatial appearance and temporal kinematics of eye blinks. The system detects a face, localizes 68 facial landmarks, extracts normalized ocular crops, and computes the Eye Aspect Ratio (EAR) for each frame. Overlapping sequences of 20 frames are processed by a dual-stream Temporal-Spatial Physiological Blink Anomaly Network (TPBAN). A lightweight MobileNetV2 encoder models local visual inconsistencies, while a bidirectional gated recurrent unit models the forward and backward dynamics of eyelid motion. Temporal attention assigns a relevance weight to every frame and supports frame-level anomaly visualization. Training uses a multi-task objective for authenticity classification and blink-phase recognition, together with a class-weighted binary cross-entropy term to address the imbalance between genuine and manipulated sequences. On the FaceForensics++ c23 test partition, BlinkNet obtained 83.16% accuracy, 87.31% ROC-AUC, 96.02% average precision, and a 21.08% equal error rate. The manipulated class achieved 0.90 precision and 0.88 recall. The implementation processed video at approximately 52 frames per second on a consumer laptop GPU and was integrated into a Flask-based forensic dashboard. These results indicate that ocular dynamics can complement spatial evidence while improving efficiency and interpretability.

G. Dhanush, D. Manikyamba · 0 citations
Conference Aug 2026

Intelligent multimodal face anti-spoofing detection system based on biometrics

With the widespread deployment of face recognition systems, high-fidelity spoofing attacks such as photo replays, video replays, and 3D masks pose significant security threats. Current face detection approaches (e.g., CNN-based methods) often fail to adequately capture the distinct properties of different modalities during feature fusion, resulting in persistent security risks within face detection systems. This paper proposes a multi-dimensional feature fusion-based face anti-spoofing detection method based on the YOLOv8 architecture. Unlike traditional methods that rely solely on static RGB input, the paper integrates dynamic optical flow features (capturing micro-movements) with static texture analysis. By utilizing a cross-scene dataset containing diverse attack samples ,the method achieves robust detection through the fusion of spatial and temporal cues. Additionally, data augmentation strategies are employed to help the model better localize and identify targets across different scales. The proposed method exhibits strong practical value for intelligent systems in complex scenarios.

Yanhua Liang, Pengcheng Zhou, Hongmei Qin et al. · 0 citations
Open access Aug 2026

Multimodal Deepfake Detection for Digital Forensics: A Robust Audio-Visual Inconsistency Approach for Evidence Integrity

The rapid proliferation of Generative AI (GenAI) has democratized the creation of hyper-realistic multimedia forgeries, posing severe threats to electronic Know Your Customer (eKYC) systems and digital forensic investigations. While visual synthesis has reached near-perfection, maintaining precise synchronization between lip movements (visemes) and speech signals (phonemes) remains a formidable challenge. To address this, we propose a novel Multimodal Deep Learning framework designed to detect high-fidelity Deepfakes by exploiting audio visual temporal inconsistencies. Beyond traditional feature fusion, our architecture integrates a Contrastive Synchronization Loss with a Transformer based Cross-Modal Attention mechanism. This hybrid objective explicitly enforces intra-class compactness for authentic pairs while amplifying the distance for asynchronous forgeries. Extensive experiments on FaceForensics++, DFDC, and a custom Vietnamese dataset (Vn-eKYC-Aug) demonstrate that our model achieves state-of-the-art performance, maintaining high robustness against video compression and environmental noise, though operational efficacy remains sensitive to extreme low-light conditions and diverse regional dialects. This research provides a resilient forensic layer for digital identity verification, ensuring evidence integrity in the GenAI era.

H. Truong, Dung The Luong, Tuan Tran · 0 citations