Skip to content

HearInContext: A Benchmark for Implicit Context in Speech Recognition

Sep 2026 · 0 citations · 28 references
Computer Science

TL;DR

HearInContext is introduced, a Mandarin-English benchmark that pairs shared synthetic speech with assistant replies supporting different interpretations that extends to explicit conditions excluded from fine-tuning and to Mandarin hotword recognition on real recordings.

Abstract

Contextual ASR can benefit from semantic cues or from target words explicitly provided in the context. We introduce HearInContext, a Mandarin-English benchmark that pairs shared synthetic speech with assistant replies supporting different interpretations. The benchmark comprises 3,764 semantic test cases built around homophones. Implicit contexts exclude candidate words; explicit contexts name the target. No-context and unrelated-context controls measure the benefit of relevant history and sensitivity to irrelevant history. Context-capable models benefit from implicit cues but achieve higher target recall with explicit hints. Fine-tuning Qwen3-ASR-1.7B improves implicit-context target recall by 11.4 percentage points in both Mandarin and English, while absolute CER/WER changes on AISHELL-1 and LibriSpeech remain below 0.1 percentage points. Gains extend to explicit conditions excluded from fine-tuning and to Mandarin hotword recognition on real recordings. Code and data are available at https://github.com/OPPO-Mente-Lab/HearInContext

View source

Similar papers

Preprint Aug 2026

Hear2Act: Benchmarking When Prosody Should Change What an Assistant Does

Hear2Act is introduced, a unified evaluation protocol for text and spoken assistants with 480 persona-grounded scenarios, hidden user concerns, and objectively verifiable outcomes that show that prosody matters when lexical evidence is insufficient, and that audio-capable LLMs can recover information from speech but do...

Xin-Yi Liu, H. Nayyeri, Dilek Hakkani-Tur et al. · 3 citations · ⚡1

SITAR: A Situational Distortion Metric for Automatic Speech Recognition Evaluation

In automatic speech recognition (ASR), transcription errors with similar lexical or semantic discrepancies can have markedly different consequences for the situation conveyed by speech. Capturing these differences requires not merely contextual assessment, but an understanding of what the spoken discourse describes, es...

Ming-Zhe Lu, Jin-Hao Liu, Yun-Peng Li et al. · 0 citations
Preprint Sep 2026

Training-Free Contextual ASR via SpeechLLM-Based Error-Aware Selective Retrieval

Recognition of domain-specific and low-frequency terms remains challenging for automatic speech recognition (ASR). Although contextual biasing can improve their recognition, directly providing a large terminology dictionary introduces many irrelevant biasing terms. Retrieval-based contextual biasing addresses this issu...

Natsuo Yamashita, Ai Nemoto, Ryosuke Koichi et al. · 0 citations

Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Models

Long-context understanding remains a fundamental challenge for large language models, as excessively long inputs often lead models to forget salient information. This issue is even more pronounced in the speech domain, where audio, as a low-compression modality, requires substantially more embeddings than text to prese...

Xi-Ze Cheng, Wen-Xu Jia, Chenyuhao Wen et al. · 4 citations · ⚡1

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.