Ju-Chieh Chou

Quantifying the Generation Modality Gap in Speech-Text Language Models

Pure speech language models often lag behind text and speech-text language models in generating coherent content, but this gap is difficult to quantify because speech and text systems are typically evaluated with different metrics and trained on different data. We study the speech-text modality gap in a family of spoke...