Dejan Stancevic

Information-Theoretic Scaling Laws for Generative Diffusion Training

Diffusion models trained on increasingly large datasets sampled from the same data distribution will converge to the same learned distribution. However, in practice, a model trained on finite data depends on the particular training dataset; this dependence is reflected in properties of the model's outputs, such as memo...