Contextual Pictograms for Repairing Multilingual ASR Errors
Mariana Hernandez
Hector Luna
Carlos Bruno
Luis Ricardo Santos
Elena Simoes
Nov 2026
Natural Language Processing
Human-computer Interaction
Abstract
Speech transcription errors can be difficult to recognize when competing word hypotheses are unfamiliar or visually similar. We describe an interface that combines alternative transcripts, replay and contextually checked pictograms for selected uncertain concepts. The system will initially support French and Spanish functional utterances. A counterbalanced study will assess whether pictorial feedback helps listeners recover the speaker's intended action beyond text alternatives alone. We will also quantify semantic mismatch and additional interaction time, since an inappropriate image can reinforce the wrong interpretation rather than repair it.
