An Energy-Efficient Sparse MobileNetV1 FPGA Accelerator With Chained Depthwise Separable Convolution and Computation-Aware Pruning

Abstract

MobileNet has been widely adopted in mobile applications due to its lightweight architecture based on depthwise separable convolution (DSC). However, developing an efficient MobileNet accelerator remains hard as the DSC consists of two heterogeneous operations, depthwise convolution (DWC) and pointwise convolution (PWC). In addition, workload ratios between the DWC and PWC vary across the DSC layers, causing workload imbalance and thus underutilization in the accelerator. Pruning that removes unimportant weights worsens this problem as sparsity is not evenly distributed. In this brief, we propose an energy-efficient FPGA-based accelerator for sparse MobileNet with a novel operation called chained depthwise separable convolution (CDSC) and computation-aware pruning (CAP). The CDSC reformulates the consecutive DWC and PWC operations as a single operation, which resolves the underutilization problem by making a fixed workload ratio between the DWC and PWC in all DSC layers. Furthermore, the CDSC combines sparsities of the DWC and PWC, and this chained sparsity significantly reduces the computations of the CDSC-based MobileNet. To maximize the effect of the chained sparsity without significant accuracy loss, the CAP iteratively searches for efficient pruning targets by simultaneously considering the accuracy and the number of computations. The proposed accelerator is implemented on the Xilinx ZCU104 FPGA platform, and achieves the best trade-off between the accuracy and energy efficiency.