Real-Time Spatio-Temporal Deepfake Detection for Live Biometric Authentication via EfficientNet-GRU
TL;DR
A highly optimized Two-Stream Spatio-Temporal architecture specifically designed for zero-latency live video evaluation, which extracts fine-grained spatial artifacts using a lightweight EfficientNet-B0 backbone, while a unidirectional Gated Recurrent Unit (GRU) models frame-to-frame physiological inconsistencies.
Abstract
Deepfake technology poses a critical threat to live video conferencing and biometric authentication. Existing detection models are either purely spatial—rendering them vulnerable to video compression—or rely on computationally heavy 3D-CNNs incompatible with strict real-time CPU latency constraints. We propose a highly optimized Two-Stream Spatio-Temporal architecture specifically designed for zero-latency live video evaluation. The framework extracts fine-grained spatial artifacts using a lightweight EfficientNet-B0 backbone, while a unidirectional Gated Recurrent Unit (GRU) models frame-to-frame physiological inconsistencies. Evaluated on a diverse 3,000-video corpus from FaceForensics++, Celeb-DF, and DFDC, the model achieved 98.21 cross-dataset accuracy and a 0.9978 AUC. Crucially, CPU inference requires only 125.63ms per 16-frame sequence—well below the 533ms threshold of a standard 30 fps camera—guaranteeing seamless, real-time overlay detection. Finally, the network's decision boundaries are mathematically validated using Explainable AI (XAI) activation maps and t- SNE clustering.