Experimental results show that the complete landmark representation provides the best overall image-generation performance among the evaluated configurations, while Post-Refinement consistently improves frame-level visual quality.
Abstract
We propose a three-stage audio-driven talking face generation framework for stable and high-fidelity facial animation. The first stage predicts an extended landmark representation from speech, the second stage synthesizes an initial talking-face sequence from the landmarks, and the final stage refines the generated frames to enhance visual realism and fine-grained appearance. The extended representation combines facial, hair-contour, and upper-body landmarks and is generated automatically for training. By separating motion prediction, image reconstruction, and appearance refinement, the framework allows each subproblem to be modeled and supervised independently. Experimental results show that the complete landmark representation provides the best overall image-generation performance among the evaluated configurations, while Post-Refinement consistently improves frame-level visual quality. The complete system also achieves favorable results in comparison with representative audio-driven talking-face methods. Overall, the proposed framework improves portrait reconstruction and perceptual quality while preserving the temporally ordered motion produced by the landmark predictor.
Audio-driven 3D facial animation has emerged as a prominent research topic in computer graphics. Existing autoregressive methods are often slow and suffer from error accumulation, whereas non-autoregressive methods often struggle to establish global contextual dependencies. Moreover, current approaches remain insuffici...
Yan-Lei Lei, Jing Chi, Chun-Lin Zhao et al.· ACM Transactions on Multimed...· 0 citations
Existing 3D talking avatar methods mainly rely on parametric face models including 3DMM or FLAME, whose compact motion representations limit their ability to synthesize extreme expressions and to model accessories such as headwear. We introduce AudioLift, a two-stage framework that uses generated multi-view videos as t...
Zhuo-Yuan Jiang, Weijie Lyu, Xiao-Yuan Wang et al.· 0 citations
Qualitative results show that the proposed dynamic style representation enables the generation of diverse speaker-dependent facial motions under different speaking styles and is well suited for applications such as virtual avatars, digital content creation, and other digital human systems.
CAS (Complementary Articulation-aware Speech Encoder), a complementary speech encoder adapted by contrastively aligning short speech clips with continuous 3D lip-mesh VQ-VAE latents, which is consistent with cross-language downstream transfer under a shared English-pretrained audio-visual prior.
Jaehong Myung, Seungho Eum, Taegon Kim et al.· 0 citations
Co-speech gesture video generation aims to synthesize realistic, audio-driven portrait videos from a single reference image and a speech signal. Despite advances in visual quality, diffusion-based approaches still suffer from degraded local details and imperfect audio-visual alignment. We present PAGE (Parts-Aware Guid...
Ya-Biao Wang, Jia-Fu Wu, Y. Xu et al.· IEEE transactions on circuit...· 0 citations
High-fidelity face normal estimation is a fundamental task for photorealistic 3D reconstruction, relighting, and avatar creation. Previous methods formulate this problem as a direct pixel-wise regression, which often leads to over-smoothed results, fails to capture fine-scale geometric details, and exhibits sensitivity...
Meng Wang, Quan-Bo Zhang, Wen-Jing Dai et al.· Proceedings of the 34th ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.