This work proposes EC\textsuperscript{2}Face, a multimodal face synthesis framework that improves semantic alignment through explicit semantic supervision and distribution-aware optimization, and introduces Explicit Conditional Consistency Guidance (ECCG), which imposes direct consistency supervision in pixel space.
Abstract
Although diffusion-based methods have substantially improved the controllability of multimodal face synthesis, their semantic alignment remains suboptimal because most existing approaches rely on implicit latent-space objectives to model the relationship between denoising variables and multimodal conditions. Such implicit modeling is often insufficient to enforce precise correspondence between synthesized faces and conditional inputs, especially under long-tailed semantic mask distributions where rare attributes receive weak optimization signals. To address these limitations, we propose EC\textsuperscript{2}Face, a multimodal face synthesis framework that improves semantic alignment through explicit semantic supervision and distribution-aware optimization. First, we introduce Explicit Conditional Consistency Guidance (ECCG), which imposes direct consistency supervision in pixel space by decoding an approximate reverse estimate of the clean latent and explicitly aligning the synthesized image with textual descriptions and semantic masks. A temporal dynamic modulation function is further designed to adapt the supervision strength according to the timestep-dependent reliability of reverse estimation. Second, we propose Long-Tail Adaptive Flow Matching (LAFM), which reweights spatial optimization signals based on semantic attribute frequency, with normalized weights to maintain numerical stability during training. Importantly, all additional modules are used only during training and introduce no extra inference overhead. Extensive experiments show that EC\textsuperscript{2}Face consistently outperforms competitive baselines in both generation quality and semantic alignment, achieving a 29.38\% improvement in mask accuracy on rare attributes.
Results show that SSGLD effectively resolves the trade-off between texture fidelity and pose-variation robustness in high-resolution paired PGPIS under sparse 2D skeleton conditioning.
This work shows that identity-conditioned face synthesis can be performed at a substantially lower computational cost by a latent Consistency Model with few iterations, without compromising image quality for large-scale synthetic face generation.
Tiago Kienen Chaves, Bernardo Biesseck, David Menotti· 0 citations
Multi subject video editing modifies designated subjects while preserving non target content, but faces cross subject attribute leakage, and occlusion ambiguity. Existing approaches rely on masks and struggle to distinguish overlapping subjects or ensure consistent generation. To address these limitations, we propose M...
A pyramid-guided multi-scale attention framework based on scale alignment and reliability-aware feature refinement improves occlusion robustness without sacrificing clean-face recognition performance, indicating its practical potential for identity verification and access-control applications involving masks, glasses,...
This work proposes a structured-prior-guided diffusion inpainting framework with physical consistency that beats seven representative competitors on every metric of reconstruction fidelity, physical consistency and semantic controllability in traffic sign detection.
Luo Li, Chong-Chong Huang, Jun Jia et al.· 0 citations
Sketch-to-Face, a cross-modal diffusion-based model that leverages Stable Diffusion to generate photorealistic faces using sketch-based inpainting, is introduced, demonstrating a favorable quality–efficiency trade-off compared with text-and-sketch diffusion baselines.
Inas Ismael Imran, Ahmed A. Hashim· Advances in Artificial Intel...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.