Where Does the Sound Go? Tracing Acoustic Information Loss in Audio-Conditioned LLMs
Audio-conditioned language models often underuse acoustic cues such as prosody, emotion, and non-speech sounds, raising the question of whether ASR-supervised frontends discard this information before it reaches the LM. We test whether the frontend is responsible by comparing Whisper-Tiny and Whisper-Small with EnCodec...
Song-ha Jo, Sehyun Lee, Soyoon Kim et al.
· 0 citations