Quantum Attention Mechanisms: Architectures, Measurement Overhead, and Empirical Evidence
TL;DR
This survey contributes a refined taxonomy with an attention-semantics dimension distinguishing pairwise from holistic mechanisms, a resource-realism analysis, and a critical synthesis of the empirical literature that finds parity or task-conditional modest gains rather than uniform quantum advantage across the five papers reporting parameter-matched classical baselines.
Abstract
Quantum attention mechanisms integrate parameterised quantum circuits or quantum linear algebra primitives into transformer architectures, drawing on superposition and entanglement in the attention computation. The field has produced more than two dozen proposals between 2022 and 2026, spanning Parameterised Quantum Circuits (PQC) on near-term hardware, Quantum Linear Algebra (QLA) on fault-tolerant hardware, and quantum annealing. This survey contributes a refined taxonomy with an attention-semantics dimension distinguishing pairwise from holistic mechanisms; a resource-realism analysis that states the measurement-overhead cost of pairwise quantum attention in the Noisy Intermediate-Scale Quantum (NISQ) regime as a proposition under explicit assumptions, with the architecture classes falling outside those assumptions identified, organised within a four-axis design framework covering sequence and qubit scaling, circuit depth, encoding overhead, and hardware platform constraints, complementing the dedicated measurement-overhead analysis; and a critical synthesis of the empirical literature that finds parity or task-conditional modest gains rather than uniform quantum advantage across the five papers reporting parameter-matched classical baselines. Six open directions are identified for the next phase of research. The survey clarifies what the current literature does and does not establish about quantum attention’s practical advantage over efficient classical alternatives.