This work presents a Burmese medical speech recognition framework built on a high-quality 28-hour corpus recorded and validated by native speakers, and fine-tune Whisper models using full fine-tuning (FFT) and parameter-efficient fine-tuning (PEFT) with LoRA.
Abstract
Although Whisper models benefit from large-scale multilingual pre-training, their performance on Burmese medical speech remains limited. This work presents a Burmese medical speech recognition framework built on a high-quality 28-hour corpus recorded and validated by native speakers. We fine-tune Whisper models using full fine-tuning (FFT) and parameter-efficient fine-tuning (PEFT) with LoRA. To evaluate robustness, we apply waveform- and spectrogram-level data augmentation under controlled noise and simulated room acoustics. While augmentation reduces performance on clean speech, it significantly improves robustness in noisy and reverberant environments across FFT and PEFT settings. Our best-performing system, fully fine-tuned myMediWhisper-Medium without augmentation, achieves a state-of-the-art Word Error Rate (WER) of 23.44%, outperforming much larger general-domain fine-tuned models. Dataset and other resources can be found at the Huggingface repository: https://huggingface.co/datasets/LULab/mediTalk-mm-rdy.
This paper investigates how multilingual medical adaptation reshapes the internal representations of Whisper models through layer-wise encoder analysis, and shows that English medical fine-tuning produces the dominant encoder shift, whereas multilingual continuation largely preserves the adapted representation space.
Souranil Kahali, Rituparna Bose, Abner Hernandez et al.· 0 citations
A controlled, fine-tuned Whisper-based Assamese ASR system trained on the Mozilla Common Voice 24.0-Assamese corpus is presented, employing mixed-precision training and gradient accumulation on Tesla 4 Graphics Processing Units (T4 GPUs).
Ganapati Das, Dwipen Laskar, Hasin Afzal Ahmed et al.· 0 citations
A preliminary study on the adaptation of Whisper for Automatic Speech Recognition in Baniwa, an indigenous Arawakan language spoken in Brazil, Colombia, and Venezuela, demonstrating that multilingual foundation models can be successfully adapted to extremely low-resource indigenous languages.
Leonardo Duart, T. Fonseca, T. Chacon· 0 citations
Despite advances in automatic speech recognition (ASR) technology, its performance remains limited when processing dysarthric speech, which hinders people with dysarthria from accessing voice-based digital technologies. In this study, we propose LoRA-MoE, a parameter-efficient adaptation method that combines Low-Rank Adaptation (LoRA) with a mixture of experts (MoE), to improve speech recognition for individuals with Parkinson’s disease (PD), who often exhibit hypokinetic dysarthria. We integrate LoRA-MoE into the Whisper model by freezing its pre-trained weights while introducing multiple LoRA experts and a router network that dynamically combines them based on the input. Experiments on a Korean PD speech dataset show that LoRA-MoE achieves a character error rate (CER) of 12.01%, a 38.54% relative improvement over the baseline Whisper model, and outperforms full fine-tuning and single LoRA adaptation by 46.00% and 11.37%, respectively, with consistent improvements across all PD severity levels. The proposed approach demonstrates consistent improvements and stable performance across all severity levels, and its performance is robust to the number of experts; we adopt three experts as our default configuration.
Seojin Yoon, Ryul Kim, Sangmin Lee· IEEE Access· 0 citations
Overall, matched-text speech delivery should be treated as a first-class factor in Audio LLM safety evaluation by holding transcript content fixed and varying six speech-delivery presets whose acoustic attributes may co-vary.
