The results imply that existing deepfake speech detection methods are insufficient for reliably detecting edit-based speech deepfakes in various scenarios unseen during training.
Abstract
Recent improvements in voice-cloning speech generation systems raise concerns about misuse by malicious actors to impersonate others and spread misinformation. Detecting such tampering is difficult, since deepfakes in the wild may be created by different generative models in a wide range of languages. Furthermore, the speech audio may also only be partially modified, presenting a different and potentially more challenging task than detecting fully-synthetic speech waveforms. To enable further research in this direction, we propose a multilingual dataset for detection and localization of partially edited speech samples. Our dataset includes speech in 12 languages, generated by up to five systems per language, and includes both a training set as well as an evaluation benchmark. To showcase the utility of our proposed dataset, we train localization models of existing architectures and study generalization across three axes: across different languages, across different speech synthesis and editing systems, and across different acoustic environments. Our results show that localization models almost always generalize poorly to speech edited by systems not seen during training. On the other hand, generalization to edited speech in unseen languages still degrades performance but to a lesser extent. We also augment our testing sets with noise to evaluate generalization across acoustic environments, and find that performance of localization models degrade significantly when tested on different acoustic conditions. All together, our results imply that existing deepfake speech detection methods are insufficient for reliably detecting edit-based speech deepfakes in various scenarios unseen during training. SPADE is publicly available on HuggingFace.
This paper introduces PersianVox, a fully automated pipeline designed to generate high-quality speech corpora from unlabeled web data, using a novel prosody-aware segmentation strategy that utilizes acoustic turn-detection to preserve linguistic completeness and optimize utterance duration for long-context modeling.
Saeedreza Zouashkiani, Soheil Khalesi, Saman Soleimani Roudi et al.· 0 citations
Real-world audio recordings for deepfake detection rarely consist solely of speech; they often are mixtures of speech and background events that themselves may also be synthetic. When processing these mixtures, it remains unclear whether detectors attend to the speech or the background event, and where their spectral e...
Recent audio deepfake detectors separate bona fide speech from synthetic speech, yet it remains unclear which stage of a text-to-speech system supplies the detection evidence. We address this with controlled resynthesis and detector adaptation in an F5-TTS-BigVGAN pipeline. Since vocoder reconstruction of a real mel ca...
Eunji Shin, Kyudan Jung, Jihwan Kim et al.· arXiv.org· 0 citations
The performance of speech classification systems such as speech emotion recognition or pathological speech detection can degrade substantially when there is a domain shift between training and test data. This shift can arise due to e.g., differences in recording setup, acoustic environment, speaker, language, or speaki...
Automatic speech recognition (ASR) performance can deteriorate considerably when speech is recorded under uncontrolled and noisy acoustic conditions, particularly for low-resource languages such as Kannada. This study presents a noise-robust ASR framework for isolated Kannada speech by integrating a previously deve...
G. P. Raghudathesh, C. Chandrakala, B. D. Rao et al.· International Journal of Spe...· 0 citations
With the rapid advancement of fine-grained speech synthesis, partial deepfake speech has emerged as an increasingly concerning threat. Recent end-to-end speech editing systems further exacerbate this challenge by enabling acoustically and perceptually seamless partial forgeries, leading to the systematic failure of exi...
Junhao Wang, Kun Wang, Li Lu et al.· Proceedings of the 34th ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.