Skip to content

Linguistic Distance Segregates Latent Representations in Automatic Speech Recognition Systems

Aug 2026 · 0 citations · 35 references
Computer Science

TL;DR

It is observed that the correlation between speakers'L1 distance and ASR error rates yields a systematic effect on English Speech, with its strength varying across datasets and models.

Abstract

While automatic speech recognition (ASR) models have achieved remarkable improvements in recent years, performance disparities persist across different speaker populations. One such disparity is for speakers whose first languages (L1) are from families distant from English. This paper investigates the relationship between first language background and English ASR performance. Through empirical analysis, we observe that the correlation between speakers'L1 distance and ASR error rates yields a systematic effect on English Speech, with its strength varying across datasets and models. This association is statistically significant in a follow-up analysis accounting for dataset-level variation in Tweedie mixed-effects models ($p<0.001$ across evaluated models). In addition, analysis of the latent space reveals a L1-based spatial segregation across deeper acoustic layers in the majority of evaluated architectures

View source

Similar papers

#natural language process... Preprint Sep 2026

Language Discrimination Improves Linguistic Learning in Multilingual Speech Models

It is shown that strengthening the model's ability to discriminate languages during pretraining reduces and, on some measures, closes this multilingual gap on continuous phonetic and higher-level linguistic measures, while preserving substantial cross-language sharing.

Maureen de Seyssel, Jie Chi, Zakaria Aldeneh · 0 citations
Preprint Aug 2026

Generative vs. Encoder Large Language Models for ASR Evaluation: A Comparative Study

The results show that encoder-based metrics remain highly competitive, while generative LLMs perform strongly in hypothesis comparison and improve the interpretability of ASR evaluation.

Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil et al. · 0 citations
#natural language process... Preprint Oct 2026

Do Speech Representations Preserve Regional Accent Across Read and Spontaneous Speech?

Regional accent cues can be captured under matched conditions, but it remains unclear whether they persist between read and spontaneous speech. We study RVG1, with 500 German speakers from nine regions, comparing ten speech representations on regional classification and continuous geolocation under matched conditions a...

P. A. Pérez-Toro, Tomás Arias-Vergara, Annette Schwarz et al. · 0 citations
#natural language process... Preprint Sep 2026

Do speech foundation models really learn words?

It is shown that this simple approach to disentanglement can enhance higher-order linguistic information in word discovery tasks and learn representations which encode words with reasonable fidelity independently of local phonetic content.

Robin Huo, Ewan Dunbar · 0 citations
#machine learning Preprint Sep 2026

A Comprehensive Study of Content Representations for Speech Synthesis

Speech content representations are central to voice conversion, speech-to-speech translation, and multimodal language models, yet they are rarely compared under a common generative framework that directly measures what each representation contains. We address this by training a generative model conditioned solely on ea...

Diego Torres, Axel Roebel, Nicolas Obin · 0 citations

Related blog posts

MIT News · Artificial Intelligence Aug 27, 2026

Looking beyond natural sequences

A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.