We present M5-TTS, a compact text-to-speech system supporting multilingual and Chinese multidialect synthesis, zero-shot voice cloning, speaker-specific multilingual generation, and CPU streaming. Its 50M-parameter language model, excluding the speech codec, combines a temporal backbone with a local multi-token predict...
Wen-Hao Li, Jing-Bin Hu, Sheng-Ming Ling et al.· 0 citations
Recent advances in speech generation are shifting toward context-aware, multi-turn, and multi-speaker synthesis. While these capabilities are in high demand for applications such as audiobook and audio drama production, evaluation has not kept pace with these advances. Beyond traditional zero-shot Text-to-Speech (TTS),...
Jing-Bin Hu, Zi-Tao Yang, Xiao-Ge Peng et al.· 0 citations
Nonverbal vocalizations (NVVs), such as laughter and sighs, convey meaning alongside words. NVV-aware speech systems must recognize and generate these events while preserving lexical accuracy and perceptual quality. Yet differences in annotation, resources, and evaluation complicate comparisons, and aggregate scores ca...
Liu-Meng Xue, Yuang Cao, Jing-Bin Hu et al.· 0 citations
Continuous latent representations underlie recent diffusion-based lyrics-to-song and text-to-music systems, yet aligning lyric conditions with these latents remains difficult: existing systems resort to generator-side machinery such as sentence-level timestamps, auxiliary alignment objectives, or semi-autoregressive de...
Guo-Bin Ma, Yue-Peng Jiang, Da-Ke Guo et al.· 0 citations
We propose a preference optimization framework with Large Audio-Language Model (LALM) feedback for controllable non-verbal vocalization (NVV) generation in continuous autoregressive speech models. To construct preference data without human preference annotation, we build a bilingual prompt corpus by combining NVV-injec...
Jing-Bin Hu, Qi-Rui Zhan, Yuang Cao et al.· 3 citations
A source-adaptive data curation strategy that refines public NVV corpora through acoustic augmentation and multimodal LLM filtering, while mining spontaneous NVVs from in-the-wild media through automated preprocessing and annotation is introduced.
Yuang Cao, Qi-Rui Zhan, Jing-Bin Hu et al.· arXiv.org· 2 citations
Recent controllable speech generation requires training data with fine-grained annotations of speaker traits, prosody, emotion, paralinguistic cues, acoustic scenes, and context. Existing workflows often rely on manual correction, paid hosted multimodal services, or fixed processing chains, which limits large-scale dat...
Qi-Rui Zhan, Shui-Yuan Wang, Jing-Bin Hu et al.· 0 citations
StreamTN is proposed, a lightweight LLM-based Chinese streaming TN model that employs a dual-track streaming framework in which input tokens and output tokens are processed on two parallel tracks, enabling low-latency real-time inference without complex prompting.
Wen-Hao Li, Jin-Rui Liang, Hao-Yu Zhang et al.· 0 citations
Progress in dialectal speech technology is hindered by the scarcity of large-scale, real-world corpora. For Minnan speech, existing resources remain limited, and few provide paired Minnan and Mandarin transcripts at scale. To address these gaps, we introduce WenetSpeech-Min, an open-source corpus comprising around 10,0...
Hao-Yu Zhang, Chun-Jiang He, Hong-Tao Li et al.· 0 citations
MuSeC is proposed, a music semantic codec that factorizes semantic and acoustic content directly from mixed signals without source separation that preserves information required for high-fidelity reconstruction while producing more LM-friendly discrete units.
Hua-Kang Chen, Guo-Bin Ma, Yue-Peng Jiang et al.· 0 citations
Dynamic speaking style adaptation in multi-turn multimodal interaction remains a major challenge for text-to-speech (TTS) systems. Existing context-aware TTS (CTTS) methods typically map dialogue context to speech in an end-to-end manner. Such implicit modeling makes contextual style decisions difficult to supervise, w...
Wen-Jie Tian, Kang-Xiang Xia, Jing-Bin Hu et al.· arXiv.org· 1 citation
SphereVAE is proposed, which constrains the VAE latent space to the unit hypersphere and regularizes the latent distribution toward a uniform prior, so that information is encoded mainly by directional variation, providing a bounded geometric target for autoregressive prediction and reducing the risk of norm drift.
Hao-Yu Zhang, Jing-Bin Hu, Han-Ke Xie et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.