Song-ha Jo

Where Does the Sound Go? Tracing Acoustic Information Loss in Audio-Conditioned LLMs

Audio-conditioned language models often underuse acoustic cues such as prosody, emotion, and non-speech sounds, raising the question of whether ASR-supervised frontends discard this information before it reaches the LM. We test whether the frontend is responsible by comparing Whisper-Tiny and Whisper-Small with EnCodec...