Skip to content
Conference Open access

Adaptive Face Region-based Vision Transformer for Video Deepfake Detection

Sep 2026 · 2026 10th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) · pp. 1457-1461 · 0 citations · 15 references

TL;DR

Attention maps consistently identify eye region, nasolabial folds, and mouth region, areas typically involved in face-swap and reenactment manipulations, providing some evidence that the model's decisions are based on meaningful facial regions, in contrast to basing them on spurious clues.

Abstract

Rapid developments in deepfake generation methods lead to their higher threat to the authenticity of digital media. Current methods such as the Convolutional Vision Transformer (CViT) perform whole-face image classification with a convolutional feature extractor trained from scratch and, thus, do not allow understanding what facial regions affect predictions. This paper introduces Adaptive Face Region-Based Vision Transformer, an approach which uses the self-attention mechanism [12] to address the stated problem. Adaptive facial regions in the form of eye, nose, and mouth area guided by facial landmarks are extracted for each frame, and a Vision Transformer pretrained on ImageNet-21k is fine-tuned on those crops instead of training a feature extractor from scratch. Frame-level decisions, each carrying a confidence score, are aggregated into a video-level decision through majority voting, and an attention map depicting the facial regions which are important for that decision is produced. Using a train/validation split with no frame leakage and 200 FaceForensics++ videos (100 real, 100 fake) split into 160 training and 40 validation videos, the model reaches 86% validation accuracy, with F1-scores of 0.85 and 0.88 for the real and fake classes respectively, and an average prediction confidence of 0.947; no independent test set was reserved in this study. Additionally, two self-recorded videos which do not belong to the training distribution were correctly classified – a real video (confidence 0.753) and a manually manipulated video (confidence 0.886). Attention maps consistently identify eye region, nasolabial folds, and mouth region, areas typically involved in face-swap and reenactment manipulations. Thus, it provides some evidence that the model's decisions are based on meaningful facial regions, in contrast to basing them on spurious clues. Furthermore, the proposed approach uses substantially less training data than the base paper: the base paper's CViT reached 91.5% accuracy while being trained on around 308,000 DFDC images. It means that transfer learning can close this gap, providing extra advantages of confidence estimation and visualization of decisions.

Read PDF

Similar papers

Review Open access Oct 2026

A Comprehensive Survey of Attention-Based Deep Learning Techniques for Robust Deepfake Face Detection

The rapid advancement of deep learning and generative artificial intelligence has significantly increased the realism of artificially generated and manipulated facial images, creating substantial challenges for digital media authenticity and security. Fake-face detection has emerged as an important research area for id...

Shivani Jatav, Dr. Shailja Shukla, Mr. Chandan Kum · 0 citations
Open access Oct 2026

Deepfake Image Detection Using Squeeze-and-Excitation Attention and Adaptive Threshold Optimization

Artificial intelligence-generated content (AIGC) has significantly improved the realism of manipulated facial images and videos, creating serious risks for social-network misinformation, content security, and digital trust. This study focuses on visual deepfake image detection rather than multimodal misinformation dete...

Chao-Ran Li · 0 citations
Open access Sep 2026

Deep Learning-Based Pre-Processing Pipeline and Hybrid Vision Transformer Model for Deepfake Video Detection

An advanced pre-processing pipeline and the detection model for improving the Deepfake detection in videos is developed and the Hybrid Localized Artifact Adaptively Weighted Multi-scale Attention Network-Spatio-Temporal Hypergraph Optimized Convolutional Vision Transformer Networks is developed for Deepfake video Detec...

Harith A. Hussein, Khalid Shaker, S. Abdullah · 0 citations
Conference Open access Oct 2026

Detection of Facial Manipulated Images using Convolutional Neural Networks

Recent advances in Generative Adversarial Network (GAN)-based image synthesis and autoencoder-driven facial manipulation have raised serious concerns about the trustworthiness of visual media. This paper presents a compact yet effective deepfake detection framework built on a Convolutional Neural Network (CNN) trained...

K. S. R. Krishna, M. S. Kasyapa, D. Dinesh et al. · 0 citations
Conference Sep 2026

A Multimodal Framework for Enhanced Deepfake Detection Using rPPG and Iris Features

Deepfakes generated by advanced deep learning models pose serious threats to privacy, public trust, and security. Conventional detection methods that rely primarily on visual artifacts are becoming less effective as synthetic media becomes increasingly realistic. This study proposes a multimodal deepfake detection fram...

Hyunkyung Lee, Tasmina Islam · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.