Skip to content
Preprint

Convergence of Denoising Diffusion Probabilistic Models: A R\'{e}nyi Divergence Perspective

Oct 2026 · 0 citations · 43 references
Computer Science Mathematics

TL;DR

This work establishes the first R\'enyi divergence convergence guarantee for critically damped DDPMs, a kinetic variant governed by critically damped Langevin dynamics, and supports the conjecture that critically damped DDPMs do not yield a theoretical improvement.

Abstract

Denoising diffusion probabilistic models (DDPMs) have emerged as a powerful framework for generative modeling by transforming standard Gaussian inputs into outputs that approximate a target distribution. While DDPMs have been proved to converge in total variation, KL divergence, and Wasserstein distance, these discrepancy measures share a fundamental limitation of being insensitive to distributional mismatch in low-probability regions. To address this, we adopt the stronger R\'enyi divergence as the discrepancy measure. Leveraging a Girsanov-type change-of-measure argument, we establish the first R\'enyi characterization of DDPMs to the best of our knowledge, showing that the total error decomposes into a mixing error, a discretization error, and a score estimation error. We assume the target distribution is sub-Gaussian and characterize the score error via the cumulant generating function, relaxing both the log-Sobolev-inequality and $L^\infty$-score-error conditions used in prior R\'enyi divergence analyses of inexact Langevin Monte Carlo, a score-based sampling algorithm closely related to DDPM. Our results also establish an iteration complexity for achieving $\epsilon$-accuracy in $\alpha$-R\'enyi divergence that is asymptotically proportional to $\alpha^3/\epsilon$. Finally, by extending our analysis, we establish the first R\'enyi divergence convergence guarantee for critically damped DDPMs, a kinetic variant governed by critically damped Langevin dynamics. Similar to existing characterizations of critically damped DDPMs, this result shares the structural scaling of standard DDPMs, differing by at most a logarithmic factor in the mixing error exponent, and supports the conjecture that critically damped DDPMs do not yield a theoretical improvement.

View source

Similar papers

#machine learning Preprint Oct 2026

Diffusion Removes Langevin's Conditioning Dependence: A Sharp Gaussian Analysis

Despite their empirical success, why diffusion models overcome the bottlenecks of classical score-based samplers remains unclear. In this work, we leverage Gaussian distributions to isolate this phenomenon. We establish 2-Wasserstein convergence bounds for optimized hyperparameters, showing that diffusion processes ach...

Adam Perbost, Francis R. Bach, Pierre Marion · 0 citations
#machine learning Preprint Oct 2026

Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models

The approach aligns the score-training objective with the sampling grid, allowing statistical estimation and numerical discretization errors to be analyzed together, and establishes finite-sample guarantees for conditional diffusion models learned from data and implemented with finitely many denoising steps.

Qiu-Yun Zhu · 0 citations
#machine learning Preprint Oct 2026

Gradient-Free Sampling from Generative Models via Stochastic Bounded Extremum Seeking

We introduce a sampling approach for energy- and score-based generative models that requires no gradient evaluations of the model. Replacing the drift term that would normally contain the score $\nabla_\mathbf{x} \log p_\theta(\bf{x})$ with a high-frequency dithered cosine of the model's \textit{value}, $\sqrt{\alpha\o...

A. Scheinker · 0 citations
#machine learning Preprint Oct 2026

Annealed L\'{e}vy Diffusion via Gaussian Surrogate Reverse Dynamics

Gaussian diffusion models offer a practical framework for generative modeling. Denoising L\'{e}vy Probabilistic Models (DLPM) introduce heavy-tailed perturbations with a fixed tail index throughout denoising. We introduce Annealed L\'{e}vy Diffusion (ALD). ALD varies the stability index $\alpha(t)$ to combine heavy-tai...

Yu-Ling Jiao, Li-Can Kang, Jerry Zhijian Yang et al. · 0 citations

Sharp Convergence and Sampling Trade-offs for Riemannian Diffusion under Nonnegative Ricci Curvature

This work develops a general framework that separates score discretization from Brownian-motion simulation and allows multiple geodesic random-walk steps per score evaluation and provides a sharper characterization of the convergence and sampling complexity of Riemannian diffusion models.

Yu-Hao Liu, Long-Bo Huang · 0 citations

Robustness of Diffusion Models under Distribution Shift

This work studies robust score estimation under Wasserstein perturbations of a reference distribution of Ornstein--Uhlenbeck diffusion and characterize how finite data, intrinsic dimension, and distribution shift affect the robustness of score-based diffusion models.

Wei Luo, N. K. Chada, Shi-Jie Zhang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.