DeStyle: A Unified Decoupled Framework for Style Transfer via Singular Value Purification and Attention-Guided Distillation

Abstract

A major challenge in training-free image style transfer is to reproduce the artistic characteristics of a reference image while preserving the semantic identity and spatial structure of the content image. Existing methods often struggle to balance content preservation and stylistic expressiveness: aggressive feature injection may introduce semantic information from the style reference, whereas conservative strategies tend to produce only coarse color and tone transfer. To address this issue, we propose DeStyle, a unified training-free framework that progressively decouples style transfer across latent initialization, feature conditioning, and latent-manifold alignment. First, Precision Trajectory Inversion (PT-Inversion) and Dual-Priors Statistical Fusion (DPSF) establish a content-aware initial latent prior by reducing inversion errors and aligning the statistical characteristics of content and style representations. Second, a dual-branch regulation architecture is equipped with Dynamic Singular Component Filtering (DSCF), which dynamically suppresses semantic interference in stylistic features through time-aware singular value filtering. Finally, Latent-Manifold Alignment via Attention-Guided Distillation (LMA-AGD) refines the latent representation by aligning attention distributions, enabling more accurate reconstruction of fine-grained artistic textures. Extensive experiments on benchmark datasets demonstrate that DeStyle achieves superior performance in structural fidelity, content-leakage suppression, and stylistic expressiveness compared with existing training-free style transfer methods.