SphereVAE is proposed, which constrains the VAE latent space to the unit hypersphere and regularizes the latent distribution toward a uniform prior, so that information is encoded mainly by directional variation, providing a bounded geometric target for autoregressive prediction and reducing the risk of norm drift.
Abstract
With the rapid development of speech generation technology, discrete codec representations have been widely used because they provide a stable prediction paradigm. In expressive speech generation, however, the quantization bottleneck of discrete codecs results in information gaps in fine-grained prosody, timbre, pronunciation, and frame-to-frame continuity. Continuous representations (e.g., VAE latents), by eliminating this constraint, have emerged as a more effective alternative for autoregressive modeling. Yet when continuous representations are used as autoregressive prediction targets, prediction errors can accumulate along the generation chain, causing latent drift and degrading long-form stability. To mitigate this problem, we propose SphereVAE, which constrains the VAE latent space to the unit hypersphere. SphereVAE defines a Power Spherical posterior on the hypersphere and regularizes the latent distribution toward a uniform prior, so that information is encoded mainly by directional variation, providing a bounded geometric target for autoregressive prediction and reducing the risk of norm drift. SphereVAE underperforms the standard VAE on reconstruction metrics due to reduced latent freedom. However, when integrated into VoxCPM for zero-shot TTS and long-text generation, it yields lower content error rates with comparable speaker similarity, and shows more stable long-range speaker consistency. These results indicate that an appropriate latent geometric constraint can effectively mitigate autoregressive error accumulation and drift in speech generation.
The proposed DiTAR+, a dual-optimization framework for continuous-latent Autoregressive Diffusion Transformer models, introduces Dilated Context Sampling and Hierarchical Acoustic Masking to prevent shallow layers from attending to acoustic pre-context, explicitly decoupling semantic alignment from acoustic detail reco...
Zi-Yu Zhang, Tian-Lun Zuo, Han-Zhao Li et al.· 0 citations
ReLMCodec is a low-bitrate single-codebook speech codec built upon a preserve--control--refine principle that moves the empirical single-stream predictability--reconstruction frontier in the evaluations, with gains that carry over to downstream text-to-speech (TTS) synthesis in both intelligibility and speaker similari...
Zixiang Wan, Xusheng Yang, Zhengmeng Wang et al.· 0 citations
On near-domain benchmarks, static and dynamic approaches perform comparably, and on harder cross-corpus benchmarks with diverse synthesis methods, trajectory dynamics provide substantial gains, confirming that temporal physiological constraints carry detection signal beyond utterance-level statistics.
CuteTTS is presented, a compact continuous-autoregressive TTS system that reconciles high-fidelity generation with the latency demands of real-time interaction and introduces guidance-step distillation, which absorbs classifier-free guidance and multiple solver steps into a single interval-conditioned student.
Yu-Qian Zhang, Yao Shi, Kexin Huang et al.· 0 citations
A unified framework covering six popular LM-based generative SE modeling paradigms based on discrete/continuous latent NAC features, and a fine-tuning strategy with auxiliary losses on reconstructed speech to improve both intrusive and non-intrusive metrics is presented.
Yihui Fu, Zhengyang Li, Tim Fingscheidt· 0 citations
This paper proposes a segmentation strategy that decomposes audio into overlapping segments, enabling a Continuous Normalizing Flow (CNF) to model the evolution of latent trajectories using Flow Matching, enabling a Continuous Normalizing Flow to model the evolution of latent trajectories using Flow Matching.
Esteban Gutiérrez, L. Wyse, F. Font et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.