Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (NFEs). We study how distributional distillation can reduce this cost by exploiting the student's probabilistic token outputs. Our unified formulation connects the student'...
Paul Le Van Kiem, Dario Shariatian, Umut Simsekli et al.· 0 citations
This work develops tighter bounds for three metrics: KL divergence, reverse KL divergence, and Wasserstein distance, effectively exploiting the regularity of the class of score estimators, and suggests that the quality of the score approximation has more impact on closing the score matching gap for low noise scales.
Benjamin Dupuis, Tyler Farghly, Maxime Haddouche et al.· arXiv.org· 0 citations
This work builds on a principled continuous-time approximation of Markov algorithms and introduces a new, exact entropy flow formula for such processes, and establishes novel connections to a well-studied family of modified logarithmic Sobolev inequalities.
Benjamin Dupuis, Maxime Haddouche, George Deligiannidis et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.