Skip to content
Open access

Stable and High-Fidelity Photo-Realistic Talking Face Generation Using a Three-Stage Model With Extended Landmarks and Post-Refinement

2026 · IEEE Access · Vol 14, pp. 162756-162771 · 0 citations · 45 references

TL;DR

Experimental results show that the complete landmark representation provides the best overall image-generation performance among the evaluated configurations, while Post-Refinement consistently improves frame-level visual quality.

Abstract

We propose a three-stage audio-driven talking face generation framework for stable and high-fidelity facial animation. The first stage predicts an extended landmark representation from speech, the second stage synthesizes an initial talking-face sequence from the landmarks, and the final stage refines the generated frames to enhance visual realism and fine-grained appearance. The extended representation combines facial, hair-contour, and upper-body landmarks and is generated automatically for training. By separating motion prediction, image reconstruction, and appearance refinement, the framework allows each subproblem to be modeled and supervised independently. Experimental results show that the complete landmark representation provides the best overall image-generation performance among the evaluated configurations, while Post-Refinement consistently improves frame-level visual quality. The complete system also achieves favorable results in comparison with representative audio-driven talking-face methods. Overall, the proposed framework improves portrait reconstruction and perceptual quality while preserving the temporally ordered motion produced by the landmark predictor.

Read PDF

Similar papers

Oct 2026

Audio-driven 3D Facial Animation with Multimodal Context Enhancement

Audio-driven 3D facial animation has emerged as a prominent research topic in computer graphics. Existing autoregressive methods are often slow and suffer from error accumulation, whereas non-autoregressive methods often struggle to establish global contextual dependencies. Moreover, current approaches remain insuffici...

Yan-Lei Lei, Jing Chi, Chun-Lin Zhao et al. · 0 citations
Preprint Oct 2026

AudioLift: Audio-Driven Single Image to Full-Head Gaussian Avatar via Multi-view Video Generation

Existing 3D talking avatar methods mainly rely on parametric face models including 3DMM or FLAME, whose compact motion representations limit their ability to synthesize extreme expressions and to model accessories such as headwear. We introduce AudioLift, a two-stage framework that uses generated multi-view videos as t...

Zhuo-Yuan Jiang, Weijie Lyu, Xiao-Yuan Wang et al. · 0 citations
Nov 2026

PersonaDiff: diffusion-based style control for speech-synchronized 3D facial animation

Qualitative results show that the proposed dynamic style representation enables the generation of diverse speaker-dependent facial motions under different speaking styles and is well suited for applications such as virtual avatars, digital content creation, and other digital human systems.

Wei Li, Yu-Pu Song, Ya-Zhi Liu · 0 citations
Preprint Oct 2026

CASE: Complementary Articulation-Aware Speech Encoder to Enhance Speech-Driven 3D Facial Animation

CAS (Complementary Articulation-aware Speech Encoder), a complementary speech encoder adapted by contrastively aligning short speech clips with continuous 3D lip-mesh VQ-VAE latents, which is consistent with cross-language downstream transfer under a shared English-pretrained audio-visual prior.

Jaehong Myung, Seungho Eum, Taegon Kim et al. · 0 citations
Open access Oct 2026

PAGE: Parts-Aware GuidancE for Co-Speech Gesture Portrait Video Generation

Co-speech gesture video generation aims to synthesize realistic, audio-driven portrait videos from a single reference image and a speech signal. Despite advances in visual quality, diffusion-based approaches still suffer from degraded local details and imperfect audio-visual alignment. We present PAGE (Parts-Aware Guid...

Ya-Biao Wang, Jia-Fu Wu, Y. Xu et al. · 0 citations
Book Open access Nov 2026

High-Fidelity Face Normal Estimation with Flow Matching

High-fidelity face normal estimation is a fundamental task for photorealistic 3D reconstruction, relighting, and avatar creation. Previous methods formulate this problem as a direct pixel-wise regression, which often leads to over-smoothed results, fails to capture fine-scale geometric details, and exhibits sensitivity...

Meng Wang, Quan-Bo Zhang, Wen-Jing Dai et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.