Hearing the Unspoken: Language Model Priors for Acoustic Adversarial Attacks
TL;DR
This new Semantic Gambit attack overcomes this causal limitation by augmenting the adversary with predictive context derived from a Large Language Model in real-time by augmenting the adversary with predictive context derived from a Large Language Model in real-time.
Abstract
Automatic Speech Recognition (ASR) systems operating in real-time settings must process acoustic input under strict temporal constraints, where transcription decisions are inherently made on incomplete information. This causal constraint serves as an information bottleneck on attackers, significantly limiting attack performance. Our new Semantic Gambit attack overcomes this causal limitation by augmenting the adversary with predictive context derived from a Large Language Model in real-time. Our experiments show that this form of augmentation can elevate the corpus-level Word Error Rate to 35.6%-a three-fold increase over the current state-of-the-art streaming attack. Ultimately, this work reveals how common, low-latency LLM tooling can be exploited to systematically subvert real-time ASR pipelines.
