Speech Tokenizers are Vulnerable: Transferable Semantic Attack and Robust Tokenizer
Speech tokenizers serve as speech front ends, encoding input speech signals into representations for downstream models, such as automatic speech recognition (ASR) systems and large audio-language models (LALMs). In this paper, we find that speech tokenizers are fragile at the semantic level: small perturbations can dis...