Energy and Area Efficient Systolic Accelerator for Sparse Neural Networks

Abstract

Deep neural networks (DNNs) are increasingly deployed in real-time applications such as autonomous driving, robotics, and computer vision, where high accuracy must be achieved under strict performance and energy constraints. These networks rely heavily on matrix multiplication operations, which impose significant computational and power demands during both training and inference. Moreover, many network parameters become zero as a result of pruning or sparsity-inducing training methods, leading to sparse matrices in which conventional hardware performs a large number of redundant multiplications and additions involving zero-valued operands. This paper presents a hardware-algorithm co-design approach in which a semi-structured pruning method and a sparsity-aware systolic accelerator are developed jointly and motivated by each other. The accelerator is based on a systolic data flow architecture configurable for matrix multiplication operations. Its processing elements (PEs) are designed to bypass zero-valued multiplications and accumulations, thereby reducing unnecessary computation and energy consumption. To further enhance zero-skipping efficiency, we perform a statistical analysis of zero distributions across different network layers. In addition, each PE employs coarse-grained execution by performing multiple multiply–accumulate (MAC) operations per cycle, which, in combination with sparsity-aware execution, improves overall throughput by eliminating redundant operations. Simulation results show that the proposed accelerator reduces the power consumption of matrix multiplication computations with 50% sparsity by 11%. Additionally, the accelerator improves the computation speed for 50% sparse matrices by 100%, while the accuracy degradation of AlexNet, considering pruning from the third layer onward, is only 6%.