End-to-end Multi-face Forgery Detection via Progressively-explicit Queries

Abstract

Deepfake technology has facilitated the spread of malicious content, creating an urgent need for efficient and accurate face forgery detection techniques. Traditional detection methods usually follow a two-stage paradigm, where faces are first detected and then classified as real or fake. However, such approaches can not simultaneously handle multiple faces, and errors in the first stage directly affect the second stage, leading to error accumulation. In this paper, we propose an end-to-end Transformer-based framework, termed Progressively Explicit Query Network (PEQNet), for multi-face forgery detection and localization. Our method extracts instance-aware activation queries from multi-scale, cross-domain features, and jointly processes them with image features through a progressively explicit query-driven decoding network to predict query-dependent masks and classes. During training, we introduce triple contrastive learning to model the mutual exclusivity among real, fake, and background regions. Additionally, we define specific real-or-fake queries to capture global class-level information, which are iteratively refined by the predicted class-aware masks and optimized by aligning them with the ground-truth. Extensive experiments demonstrate that PEQNet achieves superior performance compared with state-of-the-art methods, confirming its effectiveness for multi-face forgery detection in realistic scenarios.