Text-driven zero-shot domain adaptation for StyleGAN via CLIP guidance and stylespace regularization
Abstract
Adapting pretrained StyleGAN generators to novel domains using only text descriptions is challenging due to mode collapse and entangled feature control. We propose a zero-shot adaptation framework that operates primarily in StyleSpace, a highly disentangled control space of StyleGAN. A CLIP directional loss aligns the global style of generated images with target text prompts, while a StyleSpace consistency loss preserves source diversity through channel-level regularization based on K-means clustering of cross-domain shifts. Additionally, the disentangled nature of StyleSpace enables advanced downstream applications, including fine-grained semantic editing and multi-domain mixing, without requiring any target domain training samples. Extensive experiments demonstrate that our method outperforms existing approaches in both fidelity and diversity.