An analytical framework is developed to compare fully-connected ReLU ANNs and integrate-and-fire SNNs for time-series data with respect to their theoretical energy efficiency at matched expressive capacity, and derives an expressivity-normalized efficiency ratio and explicit thresholds in network width, spike sparsity, and ANN depth scaling.
Abstract
Spiking neural networks (SNNs) are often regarded as energy-efficient alternatives to artificial neural networks (ANNs), yet their advantage depends critically on both network architecture and data properties. We develop an analytical framework to compare fully-connected ReLU ANNs and integrate-and-fire SNNs for time-series data with respect to their theoretical energy efficiency at matched expressive capacity. By relating an inference-energy model to theoretical bounds on representational expressivity, we derive an expressivity-normalized efficiency ratio and explicit thresholds in network width, spike sparsity, and ANN depth scaling. Our analysis characterizes the regimes in which event-driven computation offsets the temporal overhead of SNNs, providing capacity-aware principles for designing energy-efficient temporal networks. It shows that ANNs exceed SNNs in expressivity-normalized efficiency only in specific regimes.
Artificial neural networks (ANN) have significantly advanced speech bandwidth extension (BWE) but suffer from high computational complexity, limiting deployment on power-constrained edge devices. While spiking neural networks (SNNs) offer an energy-efficient alternative, they often struggle to capture the long-range te...
Donghyun Kim, Sang-Ho Han, Joon-Hyuk Chang· IEEE Signal Processing Lette...· 0 citations
SpiKFAX is proposed, a second-order optimization method that formulates a computationally tractable, Kronecker-factored approximation of the Fisher information matrix specifically adapted to the structure of SNNs.
Most ANN-to-SNN conversion methods rely on a specific correspondence between the source activation and the spiking neuron dynamics. We propose a finite-state continuous-time Markov chain (CTMC) neuron framework whose stationary spike flux can approximate every continuous nonnegative monotone activation function on a co...
APEX is presented, a dual-sparsity SNN inference accelerator that integrates the PASC-IF neuron into the LoAS hardware framework, and guarantees mathematical equivalence between the converted SNN and the source ANN, thereby achieving ANN-equivalent accuracy at significantly reduced timesteps.
D. Venkatesh, S. Radhakrishnan, Rajshekhar Rakshit et al.· 0 citations
The Sparse-Activation-ReLU (SAR) layer is proposed, a single-step alternative that promotes activation sparsity without surrogate-gradient training while remaining compatible with event-based computing and is a step towards energy-efficient virtual sensing.
It is demonstrated that TNA significantly enhances classification performance across various data sets and can be applied for reducing SNNs to ternary weight precision for inference, and suggests further exploration into the application of TNA on different network architectures and data sets.
Lucas Deckers, Benjamin Vandersmissen, Ing-Jyh Tsang et al.· Neural Computation· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026