Denoising Diffusion Generative Models Secretly Calculate Attentions
It is shown that diffusion models also inherently use an attention mechanism very similar to that of transformers, and similarities in basic functional principle of auto-encoders and attention-based models allows for interchange of designs based on practical requirements.
F. Haddadi, L. Monfared, Ebrahim Rezaii et al.
· 1 citation