Skip to content
Open access

Whisper automatic speech recognition and humans are similarly sensitive to accents but use sentence context differently.

Oct 2026 · Journal of the Acoustical Society of America · Vol 160 4, pp. 3477-3487 · 0 citations · 38 references
Medicine

TL;DR

Compared with human transcribers, OpenAI's Whisper model and 75 human transcribers showed similar patterns of talker difficulty but different error patterns on isolated words, possibly due to Whisper's word-prediction model being overly restrictive in short utterances.

Abstract

Despite advancements in speech recognition technology, questions remain about model generalizability and how many models mirror human recognition. These questions are addressed by comparing OpenAI's Whisper model and 75 human transcribers on 300 English sentences (20 speakers, half female, half male, half US-accented, half Spanish-accented). Sentences ended in 100 target words, each occurring in one sensical sentence (The monkey ate the banana), one nonsensical (The monkey ate the shark), and one nonsensical + nonword (The runkey ate the shark). Target word transcription accuracy was examined both in sentences and in isolation (recordings excised from same sentences). Human transcriptions replicated existing work. There were two major findings. First, the model transcribed Spanish-accented speech less well, and it benefited from sensical contexts, particularly for Spanish-accented speech. Second, sentence context affected models differently than humans: Humans benefited only from sensical sentences (with final words in-sentence more accurate than in isolation), while Whisper benefited from any sentence context. Humans and Whisper showed similar patterns of talker difficulty but different error patterns on isolated words, possibly due to Whisper's word-prediction model being overly restrictive in short utterances. Suggestions for ASR development and the possible practical and scientific contributions of automatic speech recognition to understanding human accent processing are discussed.

Read PDF

Similar papers

Open access Sep 2026

Recognition of noise-vocoded accented speech by non-native listeners

Abstract This study examined 1) how spectral degradation and talker accent strength interact to influence non-native speech perception; and 2) how sentence context contributes to non-native listeners’ perception of spectrally degraded foreign-accented speech. Twenty-eight Mandarin-speaking English-learning adults were...

Jing Yang, Li Xu · 1 citation
Open access Nov 2026

Analysis of Deepspeech2 Transfer Learning for Balinese-Accented English Speech Recognition

Automatic speech recognition (ASR) systems are generally trained on speech corpora dominated by native speakers, resulting in higher recognition error rates for speakers with underrepresented accents. This study investigates the effect of the Balinese accent on DeepSpeech2 performance and evaluates the effectiveness of...

Pande Putu Prana Pratistha, Roy Rudolf Huizen, Dadang Hermawan · 0 citations
#natural language process... Preprint Oct 2026

Do Speech Representations Preserve Regional Accent Across Read and Spontaneous Speech?

Regional accent cues can be captured under matched conditions, but it remains unclear whether they persist between read and spontaneous speech. We study RVG1, with 500 German speakers from nine regions, comparing ten speech representations on regional classification and continuous geolocation under matched conditions a...

P. A. Pérez-Toro, Tomás Arias-Vergara, Annette Schwarz et al. · 0 citations
Nov 2026

What makes Kirara sound different from Lyney? Gender differences in off-stage conversation and mediatized speech in Japanese

Abstract This study examines gender differences in Japanese speech in two contexts: an ‘off-stage’ conversational context (radio programs) and a mediatized performative context (performances in video games and animations such as Genshin Impact and Jojo’s Bizarre Adventure). Speech from the same speakers was analysed fo...

Laura Escobar, Jessamyn Schertz · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.