Shuwei He

Let Me Look at You: Advanced Facial Expression Modeling for Conversational Speech Synthesis

This work proposes FacialTalker, a facial-expression-aware CSS framework built upon a large language model backbone, and introduces a dual direct preference optimization (DualDPO) strategy, which extends the DPO by jointly imposing preference constraints on both visual and speech token sequences, to enhance the model's...