Oct 2026· IEEE Transactions on Circuits and Systems Part 1: Regular Papers· Vol 73, pp. 6696-6709· 0 citations· 58 references
Abstract
Transformer attention mechanisms pose significant scalability challenges due to quadratic complexity in sequence length, and existing accelerators remain bottlenecked by dense arithmetic and data movement. This paper proposes CAMformer, a hardware accelerator that reinterprets attention as an associative memory operation, contributing at three levels. At the circuit level, a voltage-domain Binary Attention CAM (BA-CAM) computes Hamming similarity through analog charge sharing, achieving 1.12% mean error under PVT variation—<inline-formula> <tex-math notation="LaTeX">$7\times $ </tex-math></inline-formula> lower than time-domain approaches. At the architecture level, a three-stage pipeline with hierarchical two-stage top-<inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> filtering reduces score storage by <inline-formula> <tex-math notation="LaTeX">$8\times $ </tex-math></inline-formula> while hiding DRAM latency. At the algorithm level, this top-<inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> mechanism, co-designed with Hamming Attention Distillation (HAD), maintains <0.4% accuracy degradation on GLUE benchmarks. Implemented in 65 nm CMOS and evaluated on BERT-Large, Vision Transformer, and GPT-2 decoder workloads via HSPICE simulation and Design Compiler synthesis, CAMformer achieves 9,045 queries/mJ (<inline-formula> <tex-math notation="LaTeX">$10\times $ </tex-math></inline-formula>), 191 queries/ms (<inline-formula> <tex-math notation="LaTeX">$4\times $ </tex-math></inline-formula>), and 0.26 mm<sup>2</sup> (6–<inline-formula> <tex-math notation="LaTeX">$8\times $ </tex-math></inline-formula> reduction) for attention computation compared to state-of-the-art accelerators. These results demonstrate that reconceptualizing attention as associative memory retrieval enables order-of-magnitude efficiency gains for edge Transformer inference.
This work presents a multi-role unified buffer that dynamically reuses seven banks as input, weight, intermediate, score, and output storage that reduces BRAM36 requirements by 56.3%, while buffer-control and bank-reconfiguration logic accounts for 4.3% of the lookup tables and 5.3% of the flip-flops in the full accele...
Wooyoung Park, Sangbo Park, Hyungwon Kim· IEEE Access· 0 citations
WMHA is exploited in WMHA, a latency-first diffusion-transformer inference accelerator: a very-long-instruction-word sequencer issues four engines from one instruction word, a weight-stationary 16x16 dual-dot array streams FP8 and BF16 contractions, and a single-pass online-softmax attention pipeline keeps keys and val...
This work proposes a Precision-Reconfigurable Zero-Skipping SRAM-CIM (PRZS-CIM) architecture that combines bit-serial exact multiply-accumulate decomposition, 4/6/8-bit precision control, row-level zero gating, and all-zero bit-plane suppression.
Bitla Prabhakar T. Satyanarayana, Dr. Malothu Amru, Dr. Somala Rama Kishore· International Journal of Adv...· 0 citations
Machine learning inference is moving to the edge due to latency, energy, and privacy constraints. The growing complexity of edge workloads places increasing pressure on memory hierarchies, making data movement a major limitation for both performance and energy efficiency. Fixed-function accelerators inherit the memory...
A memory-efficient hardware accelerator for depthwise separable convolution that minimizes off-chip memory traffic and parameter storage and performs the depthwise separable convolution with a small number of logic resources and on-chip memory, confirming its feasibility for resource-constrained edge devices.
Jaeseong Kim, Taehong Min, Chaebin Lee et al.· Electronics· 0 citations
Memory-polynomial digital predistortion evaluates a small fixed coefficient set over a sliding input history, so its reduction step is a complex-MAC workload with local reuse, so its reduction step is a complex-MAC workload with local reuse.
Takuto Ando, Yasuhiko Nakashima· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comOct 29, 2026
Host Sowbhagya Suresh shares some of the latest news and cutting-edge research from across Microsoft—from understanding the challenges facing coding agents to uncovering how people use AI at work and improving extended-range weather forecasting. The post Microsoft Research Focus – Edition #97 appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduOct 28, 2026
Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.