This work establishes the first R\'enyi divergence convergence guarantee for critically damped DDPMs, a kinetic variant governed by critically damped Langevin dynamics, and supports the conjecture that critically damped DDPMs do not yield a theoretical improvement.
Abstract
Denoising diffusion probabilistic models (DDPMs) have emerged as a powerful framework for generative modeling by transforming standard Gaussian inputs into outputs that approximate a target distribution. While DDPMs have been proved to converge in total variation, KL divergence, and Wasserstein distance, these discrepancy measures share a fundamental limitation of being insensitive to distributional mismatch in low-probability regions. To address this, we adopt the stronger R\'enyi divergence as the discrepancy measure. Leveraging a Girsanov-type change-of-measure argument, we establish the first R\'enyi characterization of DDPMs to the best of our knowledge, showing that the total error decomposes into a mixing error, a discretization error, and a score estimation error. We assume the target distribution is sub-Gaussian and characterize the score error via the cumulant generating function, relaxing both the log-Sobolev-inequality and $L^\infty$-score-error conditions used in prior R\'enyi divergence analyses of inexact Langevin Monte Carlo, a score-based sampling algorithm closely related to DDPM. Our results also establish an iteration complexity for achieving $\epsilon$-accuracy in $\alpha$-R\'enyi divergence that is asymptotically proportional to $\alpha^3/\epsilon$. Finally, by extending our analysis, we establish the first R\'enyi divergence convergence guarantee for critically damped DDPMs, a kinetic variant governed by critically damped Langevin dynamics. Similar to existing characterizations of critically damped DDPMs, this result shares the structural scaling of standard DDPMs, differing by at most a logarithmic factor in the mixing error exponent, and supports the conjecture that critically damped DDPMs do not yield a theoretical improvement.
Despite their empirical success, why diffusion models overcome the bottlenecks of classical score-based samplers remains unclear. In this work, we leverage Gaussian distributions to isolate this phenomenon. We establish 2-Wasserstein convergence bounds for optimized hyperparameters, showing that diffusion processes ach...
Adam Perbost, Francis R. Bach, Pierre Marion· 0 citations
The approach aligns the score-training objective with the sampling grid, allowing statistical estimation and numerical discretization errors to be analyzed together, and establishes finite-sample guarantees for conditional diffusion models learned from data and implemented with finitely many denoising steps.
We introduce a sampling approach for energy- and score-based generative models that requires no gradient evaluations of the model. Replacing the drift term that would normally contain the score $\nabla_\mathbf{x} \log p_\theta(\bf{x})$ with a high-frequency dithered cosine of the model's \textit{value}, $\sqrt{\alpha\o...
Gaussian diffusion models offer a practical framework for generative modeling. Denoising L\'{e}vy Probabilistic Models (DLPM) introduce heavy-tailed perturbations with a fixed tail index throughout denoising. We introduce Annealed L\'{e}vy Diffusion (ALD). ALD varies the stability index $\alpha(t)$ to combine heavy-tai...
Yu-Ling Jiao, Li-Can Kang, Jerry Zhijian Yang et al.· 0 citations
This work develops a general framework that separates score discretization from Brownian-motion simulation and allows multiple geodesic random-walk steps per score evaluation and provides a sharper characterization of the convergence and sampling complexity of Riemannian diffusion models.
This work studies robust score estimation under Wasserstein perturbations of a reference distribution of Ornstein--Uhlenbeck diffusion and characterize how finite data, intrinsic dimension, and distribution shift affect the robustness of score-based diffusion models.
Wei Luo, N. K. Chada, Shi-Jie Zhang et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.