Skip to content

CAMformer: Binary Associative Memory Is All You Need

Oct 2026 · IEEE Transactions on Circuits and Systems Part 1: Regular Papers · Vol 73, pp. 6696-6709 · 0 citations · 58 references

Abstract

Transformer attention mechanisms pose significant scalability challenges due to quadratic complexity in sequence length, and existing accelerators remain bottlenecked by dense arithmetic and data movement. This paper proposes CAMformer, a hardware accelerator that reinterprets attention as an associative memory operation, contributing at three levels. At the circuit level, a voltage-domain Binary Attention CAM (BA-CAM) computes Hamming similarity through analog charge sharing, achieving 1.12% mean error under PVT variation—<inline-formula> <tex-math notation="LaTeX">$7\times $ </tex-math></inline-formula> lower than time-domain approaches. At the architecture level, a three-stage pipeline with hierarchical two-stage top-<inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> filtering reduces score storage by <inline-formula> <tex-math notation="LaTeX">$8\times $ </tex-math></inline-formula> while hiding DRAM latency. At the algorithm level, this top-<inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> mechanism, co-designed with Hamming Attention Distillation (HAD), maintains <0.4% accuracy degradation on GLUE benchmarks. Implemented in 65 nm CMOS and evaluated on BERT-Large, Vision Transformer, and GPT-2 decoder workloads via HSPICE simulation and Design Compiler synthesis, CAMformer achieves 9,045 queries/mJ (<inline-formula> <tex-math notation="LaTeX">$10\times $ </tex-math></inline-formula>), 191 queries/ms (<inline-formula> <tex-math notation="LaTeX">$4\times $ </tex-math></inline-formula>), and 0.26 mm<sup>2</sup> (6–<inline-formula> <tex-math notation="LaTeX">$8\times $ </tex-math></inline-formula> reduction) for attention computation compared to state-of-the-art accelerators. These results demonstrate that reconceptualizing attention as associative memory retrieval enables order-of-magnitude efficiency gains for edge Transformer inference.

View source

Similar papers

Open access 2026

A Multi-Role Unified Buffer Architecture for an On-Device Vision Transformer Accelerator

This work presents a multi-role unified buffer that dynamically reuses seven banks as input, weight, intermediate, score, and output storage that reduces BRAM36 requirements by 56.3%, while buffer-control and bank-reconfiguration logic accounts for 4.3% of the lookup tables and 5.3% of the flip-flops in the full accele...

Wooyoung Park, Sangbo Park, Hyungwon Kim · 0 citations
Preprint Sep 2026

The World Model Hardware Accelerator

WMHA is exploited in WMHA, a latency-first diffusion-transformer inference accelerator: a very-long-instruction-word sequencer issues four engines from one instruction word, a weight-stationary 16x16 dual-dot array streams FP8 and BF16 contractions, and a single-pass online-softmax attention pipeline keeps keys and val...

Shashank Chaurasia · 0 citations
Open access Sep 2026

Precision-Reconfigurable Zero-Skipping SRAM Compute-in-Memory Architecture for Energy-Efficient Edge-AI Acceleration

This work proposes a Precision-Reconfigurable Zero-Skipping SRAM-CIM (PRZS-CIM) architecture that combines bit-serial exact multiply-accumulate decomposition, 4/6/8-bit precision control, row-level zero gating, and all-zero bit-plane suppression.

Bitla Prabhakar T. Satyanarayana, Dr. Malothu Amru, Dr. Somala Rama Kishore · 0 citations
Open access Oct 2026

QUINN: Quantized Inference and Dataflow Characterization for Neural Networks on RISC-V Near-Memory Architectures

Machine learning inference is moving to the edge due to latency, energy, and privacy constraints. The growing complexity of edge workloads places increasing pressure on memory hierarchies, making data movement a major limitation for both performance and energy efficiency. Fixed-function accelerators inherit the memory...

Vincenzo Petrolo, Flavia Guella, Guido Masera et al. · 0 citations
Open access Aug 2026

A Memory-Efficient Depthwise Separable Convolution Accelerator Using Run-Length Coding

A memory-efficient hardware accelerator for depthwise separable convolution that minimizes off-chip memory traffic and parameter storage and performs the depthwise separable convolution with a small number of logic resources and on-chip memory, confirming its feasibility for resource-constrained edge devices.

Jaeseong Kim, Taehong Min, Chaebin Lee et al. · 0 citations
Preprint Sep 2026

Energy-Oriented CGLA Mapping of a Memory-Polynomial Digital Predistortion Kernel

Memory-polynomial digital predistortion evaluates a small fixed coefficient set over a sliding input history, so its reduction step is a complex-MAC workload with local reuse, so its reduction step is a complex-MAC workload with local reuse.

Takuto Ando, Yasuhiko Nakashima · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Oct 29, 2026

Microsoft Research Focus – Edition #97

Host Sowbhagya Suresh shares some of the latest news and cutting-edge research from across Microsoft—from understanding the challenges facing coding agents to uncovering how people use AI at work and improving extended-range weather forecasting. The post Microsoft Research Focus – Edition #97 appeared first on Microsoft Research.

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.