Speech large language models (LLMs) are often built using pretrained acoustic encoders as an audio interface to an LLM decoder. However, LLMs scale poorly with input sequence length, while acoustic encoders tend to produce substantially longer sequences than typical textual inputs. This motivates learning to compress e...
Ci-Han Xiao, Yuan Tseng, Matthew Wiesner et al.· 0 citations
The results imply that existing deepfake speech detection methods are insufficient for reliably detecting edit-based speech deepfakes in various scenarios unseen during training.
Yuan Tseng, Aishwarya Fursule, Andrew Zijun Ma et al.· 0 citations
The proposed Speaker Handles provide an efficient way to integrate acoustic speaker identity into frozen text LLMs for speaker-content reasoning, and are presented as a controlled shared-agent benchmark in which overlapping facts across users require correct cross-session speaker attribution.
Run-Qiu Xu, Zhi-Sheng Zheng, David Harwath· arXiv.org· 0 citations
This paper presents a scalable method that measures consonant contribution using acoustic masking, and relates MMR to two linguistic factors previously reported to correlate with consonant contribution, namely phoneme frequency and functional load.
Eunjung Yeo, Kwanghee Choi, K. Kothadia et al.· 0 citations
Multiparty Bench is introduced, the first benchmark specifically designed to objectively evaluate conversational speech systems as active participants within multi-party contexts and assesses agent behavior along two primary dimensions: turn-taking awareness and response appropriateness.
Yi-Jen Shih, Shih-Yun-Shan Kuan, Guan-Ting Lin et al.· Conference on Empirical Meth...· 2 citations· ⚡1
Experimental results show that CtrlSpeech achieves competitive zero-shot TTS performance and improves controllability over expressive attributes, demonstrating its effectiveness for flexible and practical expressive speech synthesis.
Zhi-Sheng Zheng, Xiaohang Sun, Zhu Liu et al.· 0 citations
Speech large language models (SpeechLLMs) offer reduced latency and retain paralinguistic nuances that are typically lost in cascaded automatic speech recognition (ASR) and text-based LM architectures. However, they continue to lag behind text-only LLMs on complex reasoning tasks, while real-time spoken interaction imp...
Yi-Jen Shih, Pu-Yuan Peng, Abdel-rahman Mohamed et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.