This work proposes Agentic-GER, an LLM-based agent for terminology correction in long-form speech, which uses global context from the full transcript to identify suspicious terms and resolve ambiguous hypotheses.
Abstract
Recent advances in speech language models have improved automatic speech recognition (ASR) for long-form audio. However, accurately and consistently transcribing domain-specific terminology remains challenging. Motivated by the world knowledge and contextual capability of large language models (LLMs), we propose Agentic-GER, an LLM-based agent for terminology correction in long-form speech. The agent uses global context from the full transcript to identify suspicious terms and resolve ambiguous hypotheses. It selectively re-transcribes the source speech to check candidate corrections, and uses accepted edits to guide subsequent decisions. Experiments with four LLMs and two ASR systems on GigaSpeechBench show consistent terminology improvements in both Chinese and English, with and without thinking. On Chinese speech, Agentic-GER achieves up to a 36.8% relative reduction in biased character error rate (B-CER) over the Whisper baseline.
EviSI, a large language model evaluation agent combining Multidimensional Quality Metrics (MQM) with criteria developed with professional interpreters, is proposed, showing positive system ranking correlations with COMET throughout.
Ben Yan, Zong-Yao Li, Xiao-Yu Chen et al.· 0 citations
Voice agents depend on accurate automatic speech recognition (ASR) to act on what callers say, yet ASR is evaluated on read, English-centric speech with word error rate (WER), which penalizes surface rather than semantic differences. We introduce mu-bench, a dataset of 4,270 caller utterances from 250 phone calls to an...
This work investigates ESPnet-SpeechLM as a token-based backbone for generating SD hypotheses, formulating SD as autoregressive generation of structured tokens conditioned on acoustic input and shows that outputs generated by SpeechLMs encode useful temporal SD structure, but full-meeting SD remains limited by recordin...
HearInContext is introduced, a Mandarin-English benchmark that pairs shared synthetic speech with assistant replies supporting different interpretations that extends to explicit conditions excluded from fine-tuning and to Mandarin hotword recognition on real recordings.
Localized Attention-Constrained Inference (LACI), an inference-only method to detect TTS errors in near real-time, roll back to the error onset and regenerate with temporary guardrails, adding negligible computational overhead to demonstrate the efficacy of LACI on voice cloning reliability.
Rong-Xiang Wang, Berkin Durmus, Aysegul Orhon et al.· 0 citations
This work refine warm-initialized LLM-based ASR models by leveraging their own pre-adaptation base LLMs by leveraging their own pre-adaptation base LLMs, focusing on LoRA-adapted settings where the base LLM is preserved.
Chan-Jan Hsu, Jaeyeon Kim, Chao-Han Huck Yang et al.· 0 citations