Review
Aug 2026
Not All Attention Is Equal: A Quantitative Survey of the EEI Trade-off
This survey traces attention from Bahdanau-Luong alignment through the Transformer and into vision architectures, and reviews fixed and learned sparse attention, linear attention, IO-aware exact algorithms including FlashAttention, and state-space alternatives including Mamba.
Aditya Singh
· 0 citations