Skip to content
Preprint

EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis

Aug 2026 · 6 citations · 80 references
Engineering Computer Science

TL;DR

A multi-pass flow blending pipeline synthesizes frame-aligned transition audio, circumventing the scarcity of natural intra-utterance transitions, and dual-stage Valence-Arousal-Dominance conditioning guides prosodic planning in the LLM and acoustic realization in the flow decoder via frame-level VAD embeddings.

Abstract

Psychological research on emotion dynamics has established that human affect is a continuous, evolving process: emotions rise, decay, and transition within seconds. Current emotional text-to-speech (TTS) systems, however, condition on a single discrete label or static embedding per utterance, fundamentally misaligning with the temporal nature of affect. While recent LLM-based TTS systems may implicitly vary prosody through text understanding, such variation is neither explicitly controllable nor precise enough for targeted intra-utterance transitions. We address three challenges: (1) a multi-pass flow blending pipeline synthesizes frame-aligned transition audio, circumventing the scarcity of natural intra-utterance transitions; (2) dual-stage Valence-Arousal-Dominance (VAD) conditioning guides prosodic planning in the LLM and acoustic realization in the flow decoder via frame-level VAD embeddings; (3) direction-magnitude decoupled injection structurally separates emotion direction from injection magnitude, preventing content degradation. EmoTra-TTS adds only +0.43% parameters with no latency overhead, achieves 30%-87% relative improvement on emotion transition quality, corroborated by 64.4%-79.5% overall win rates in pairwise preference tests against four SOTA baselines and two commercial systems.

View source

Similar papers

#machine learning Preprint Aug 2026

Dual-Scale State-Space Modeling with Speaker-Wise Dynamic CRF for Speech Emotion Recognition in Conversation

DSSM-CRF, an audio-only architecture that explicitly separates cross-speaker contextual influence and within-speaker emotion evolution, is proposed and matched controls demonstrate complementary gains from speaker-wise factorization and CRF modeling.

Guang-Hua Wen, Hou-Chiang Tseng, Kuan-Yu Chen · 0 citations
Preprint Sep 2026

Post-Training Zero-Shot TTS for Fine-Grained Emotion and Duration Control via Natural Language

A unified post-training framework that equips pretrained text-to-speech models with natural-language control over segment-level emotion and duration is proposed, highlighting post-training as a practical approach to extending existing speech synthesis models.

Lian-Ru Gao, Yu-Jie Guo, Yong Qin · 0 citations
#machine learning Preprint Oct 2026

Steerspeech: Activation Steering For Emotion Control In Generated Speech

SteerSpeech is presented, a lightweight activation-steering framework that controls emotion by injecting steering vectors into hidden activations that encourages monotonic emotion control while preserving speaker identity and linguistic content, constraining steering drift, and keeping the TTS backbone frozen.

A. Benazir, Darius Petermann, Felix Xiaozhu Lin et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.