A comparative analysis of pretrained Wav2Vec XLSR-53 and Whisper-Small models for automatic speech recognition in the Telugu language

Abstract

This study presents the development of an automatic speech recognition (ASR) system tailored for Telugu, one of the widely spoken Indian languages. In recent years, deep learning (DL) techniques have been applied to develop ASR systems across various languages and domains. These models, however, require substantial training resources and extensive corpora of continuous speech composed from multiple dialectal speakers, along with their corresponding transcripts. This paper investigates the effectiveness of pre-trained models like Wav2Vec XLSR-53 and Whisper-Small for developing ASR systems for the Telugu language, addressing the challenge of limited data availability and demonstrating satisfactory results even when fine-tuned on a smaller dataset. We utilized approximately 20 h of speech data comprising 17,421 sentences of the Telugu language. The models are fine-tuned on four publicly available datasets, including OpenSLR, Common Voice, IndicVoices, and IndicTTS, to introduce greater diversity in both speaker demographics and linguistic content. The Wav2Vec XLSR-53 model achieved a word error rate (WER) of 27.3% and a character error rate (CER) of 6.8% on the test dataset, whereas the Whisper-Small attained a WER of 28.67% and a CER of 7.55%. In addition, performance of the models was evaluated by introducing noise to both individual datasets as well as a combined noise dataset. The results show that, on the combined noise dataset, Wav2Vec XLSR-53 achieved a WER of 19.59% and a CER of 4.58%, while Whisper Small obtained a lower WER of 13.97% and a CER of 3.54%. These results underscore the usefulness of leveraging pre-trained architectures in low-resource linguistic scenarios such as Telugu.