Preprint

Evaluating Pre-trained Speech Encoders for Spontaneous Speech Detection and Out of Domain Synthetic Speech Generalisation in Indic Languages

Varun Rai J. PavanKumar Sujith Pulikodan Nihar Desai

Aug 2026 · 0 citations · 28 references

Engineering

TL;DR

Centroid analysis shows that out-of-domain generalisation is predicted by a training system's proximity to unseen TTS embeddings, not its distance from natural speech, a finding with direct implications for training data selection in real-world deepfake detectors.

Abstract

Transformer-based models have shown strong accuracy in distinguishing spontaneous from scripted speech and natural from synthetic speech, but these results are established on a narrow set of well-resourced language benchmarks and have not been extended across Indic languages, nor has embedding geometry been used to explain encoder behaviour or deepfake generalisation failure. We address these gaps by evaluating five frozen transformer encoders, AST, Vaani-FastConformer, Wav2vec2, Whisper and BEATs, across 22 Indic languages, and by conducting a multi-system TTS generalisation experiment across four TTS models. Beyond accuracy, we present language isolation probing and centroid proximity analysis. Probing reveals an encoder-dependent trade-off between language-discriminability and spontaneity detection. Centroid analysis shows that out-of-domain generalisation is predicted by a training system's proximity to unseen TTS embeddings, not its distance from natural speech, a finding with direct implications for training data selection in real-world deepfake detectors.

arXiv Semantic Scholar

View source

Similar papers

Jul 2026

Large Audio Language Models for Spoofing-Aware Speaker Verification

This work systematically evaluates LALMs for SASV against conventional pipelines under zero-shot prompting, supervised adaptation, reasoning-oriented training, and reinforcement-learning-based optimization, and finds that competitive SASV performance can be achieved through several distinct routes.

Sofya Savelyeva, Mariia Perunova, E. Kushnir et al. · arXiv.org · 0 citations

Jul 2026

Content is What Remains: Invariant Speech Tokenization from Parallel Utterances

PINT (Parallel INvariant Tokenization), which fine-tunes an SSL encoder with alignment losses across parallel utterances and augmentations to distill this shared residual of linguistic content, and shows a 98.7% relative reduction in speaker probe accuracy.

Laurin Wagner, Bernhard Thallinger, Miroslav Stankovič et al. · arXiv.org · 2 citations

Jul 2026

ParaASR: Multi-Token Prediction for Fast and Long-Context LLM-Based Speech Recognition

ParaASR is introduced, an ASR system that leverages Multi-Token Prediction (MTP) to let a 4B LLM decoder emit multiple tokens per forward step and shows that decoder scaling, low-latency inference, and long-context transcription need not be competing goals when future-token proposals are anchored by the acoustic signal...

Qing-Jian Lin, Yuxin Li, Haoyang Zhang et al. · arXiv.org · 2 citations

Open access Jul 2026

A comparative analysis of pretrained Wav2Vec XLSR-53 and Whisper-Small models for automatic speech recognition in the Telugu language

This paper investigates the effectiveness of pre-trained models like Wav2Vec XLSR-53 and Whisper-Small for developing ASR systems for the Telugu language, addressing the challenge of limited data availability and demonstrating satisfactory results even when fine-tuned on a smaller dataset.

J. Pushparaj, Muzaffar Ahmad Dar, Sri Gani Kaarthikeya Kammula et al. · Frontiers in Artificial Inte... · 0 citations

Jul 2026

DONDO: Open w2v-BERT Speech-Recognition Base Models for African Languages

We present DONDO, a family of open, permissively licensed automatic speech recognition (ASR) base models for African languages, built on the w2v-BERT 2.0 self-supervised speech encoder. DONDO comprises twenty-one monolingual models and five multilingual models spanning twenty-seven language varieties across Ghana, Sier...

Paul Azunre, Naafi Dasana Ibrahim, Joel Budu et al. · arXiv.org · 0 citations

Jul 2026

Do Speech Tokens Leak Voiceprints? Speaker Inversion Attacks Against End-to-End Speech Language Models

This work introduces Audio BERT (AuB), a trainable model that constructs token embeddings from discrete codebooks and aggregates them into speaker-sensitive representations, and proposes SpInv, a two-stage inversion method built on AuB to recover embeddings in the space of an attacker-specified speaker encoder.

Ye Lu, Yi-Han Yan, Zhaoyang Zhang et al. · arXiv.org · 0 citations