Kartik Soni, Anmol Tyagi, Dr. Vishal Shrivastava, Dr. Akhil Pandey, Dr. Vibhakar Pathak

Spiking Transformers and Event-Driven Foundation Models: Surrogate Gradient Dynamics, Spike-Driven Attention, and Neuromorphic Silicon Benchmarks

Standard digital Transformer architectures rely on dense matrix-matrix multiplications (GEMM) characterized by continuous-valued activations, generating massive thermal energy footprints and memory bandwidth bottlenecks during inference and training. In contrast, biological brains process complex spatio-temporal inform...