Diffusion models achieve impressive image-generation quality but remain expensive at inference time. Diffusion distillation reduces sampling steps, yet many distilled models, including SDXL-Lightning and distribution matching distillation methods, suffer from degraded Fr\'echet Inception Distance (FID). We analyze this phenomenon through a PAC-style generalization bound. Our analysis suggests that aggressive early-step redirection of the velocity field makes the distillation target harder to learn, enlarging the train-test gap. As a result, early-step output distributions differ between training and inference, causing distribution mismatch in the intermediate noisy latent used as next-step inputs. We empirically validate this mechanism by showing reduced diversity in both intermediate features and final outputs. To address this issue, we propose EMPURPLE, a simple training-free method that recycles intermediate latents sampled from the original model. EMPURPLE is model-agnostic and improves FID by 7\% to 20\% across DMD2, Hyper-SD, FlashSD, and SDXL-Lightning. The repo is: https://github.com/TheLovesOfLadyPurple/Empurple-Training-Free-Algorithm-To-enhance-Diversity-of-The-Diffusion-Distillation-Model
The Interval Denoiser, a theoretically rigorous framework for latent-free generation, derived directly from the flow matching ODE, establishes an exact analytical mapping for intermediate trajectory states and is shown to reside on a low-dimensional manifold across any time interval.
A.M. Zaytsev, Dmitry Baranchuk, Alexander Korotin et al.· 0 citations
Diffusion models have demonstrated remarkable performance across a wide range of generative tasks; however, their high sampling cost remains a critical bottleneck. To address this, consistency distillation (CD) was proposed, offering a reduction in sampling cost by distilling a pretrained diffusion model. However, achieving generative quality comparable to diffusion models requires extensive training for the distillation process, posing a substantial computational challenge. In this article, we introduce variance-reduced consistency learning (vrCL), a novel distillation technique that enables stable and efficient training of consistency models without relying on teacher model evaluations. By leveraging a student-guided sample pair, vrCL ensures training stability while significantly reducing computational costs. This design eliminates the need for repeated teacher model evaluations during training, resulting in high computational efficiency and significantly reduced training time. Empirical results demonstrate that vrCL achieves competitive generative performance with high training efficiency, reaching strong results within just 100k training iterations.
Seer is proposed, a training-free framework that detects their valid semantic boundary using a Signal-to-Noise Ratio (SNR)-based criterion and performs one-shot truncation of the redundant suffix for all subsequent computations, offering a highly efficient, plug-and-play solution for DMLLM acceleration.
A novel post-training acceleration framework that exploits this redundancy by integrating dynamic structural sparsification directly into the distillation process, and introduces a Progressive Training Strategy coupled with an Output Rollout Mechanism that ensures the coherent learning of structural decisions across timesteps.
Yu Cheng, Siyue Yao, Zhongang Qi et al.· 0 citations
This paper forms few-step generation as a controlled base generative process, and shows that self-consistency loss can be understood through the lens of optimal control, and draws a connection between this approach and reinforcement learning, potentially opening the door to a new set of approaches for few-step generation.
Paribesh Regmi, S. Ghimire, Rui Li· International Conference on...· 0 citations
Inference-time quality-enhancement methods are an effective and widely adopted means of improving diffusion models without expensive retraining. We study a family of training-free techniques conceptually rooted in Classifier-Free Guidance (CFG), most of which were originally proposed on older U-Net diffusion models and validated using metrics that assess image quality in isolation, without accounting for compositional alignment or semantic correspondence between the generated image and its associated text prompt. We re-evaluate eight such methods on two open-weight rectified-flow transformers under a fixed per-model protocol and three compositional-alignment benchmarks. No method consistently improves on CFG across the measured criteria. APG obtains several nominal best scores, but the corresponding gains often remain within the estimated evaluation uncertainty. Attention-perturbation methods provide isolated gains on SD3.5 Medium and more frequent degradations on FLUX.2 [klein] 4B Base, while CFG remains a competitive lower-cost baseline.
A. Sergievskii, Artyom Turevich, Sergey Kastryulin· 0 citations