SparsePixels: Efficient Convolution for Sparse Data on FPGAs

A Memory-Efficient Depthwise Separable Convolution Accelerator Using Run-Length Coding

On edge devices, convolutional neural network (CNN) inference is bottlenecked mainly by memory bandwidth, owing to the frequent memory accesses to feature maps and parameters. To address this challenge, we propose a memory-efficient hardware accelerator for depthwise separable convolution that minimizes off-chip memory traffic and parameter storage. The proposed architecture employs three key techniques: (1) a 64-bit run-length coding (RLC) packet compression that exploits feature-map sparsity after ReLU, (2) a mixed-precision scheme that represents feature maps and weights at different precisions, and (3) separated depthwise and pointwise convolution units. In particular, feature maps are transferred in RLC-compressed form, which reduces the amount of data exchanged with the host. The compressed data are decoded row by row, so the on-chip buffers hold only the rows required for computation rather than a complete feature map. In software simulation on ImageNet, the mixed-precision scheme reduced the parameter storage by 49.22% at a cost of 6.24 percentage points (pp) in Top-1 accuracy, and the RLC reduced the data by up to 56.07% in the deeper layers. Implemented on a Xilinx ZCU-104 FPGA, the proposed accelerator performs the depthwise separable convolution with a small number of logic resources and on-chip memory, confirming its feasibility for resource-constrained edge devices.

A Low-Power Sparse Convolution Accelerator with Idle-First-Task-Assignment for Edge Vision

In recent years, edge-vision monitoring systems for applications such as smart animal husbandry have faced strict tripartite constraints: maintaining input resolution under extremely limited transmission bandwidth and strict power budgets. Conventional dense convolutional neural networks (CNNs) cannot satisfy the resource limits of such constrained IoT nodes. To address this challenge, this paper presents a low-power sparse convolution accelerator for edge devices, fabricated and validated in a 16 nm process. First, the accelerator adopts a bitmap-based format for compression in both data transmission and computation, effectively reducing memory and bandwidth overhead. Second, to mitigate load imbalance in sparse computation, an Idle-First-Task-Assignment (IFTA) dynamic scheduling strategy is proposed, significantly reducing processing-element (PE) idle time and improving multiplier utilization. In addition, a dedicated dataflow is designed to support and accelerate depthwise separable convolution (DWConv), which is widely used in lightweight networks. Experimental results show that the chip occupies only 0.5~mm$^2$ core area and consumes as little as 12--16~mW. On ImageNet, for sparse VGG16 and MobileNetV2, the proposed accelerator achieves 6.5$\times$ and 2.8$\times$ speedups, respectively, over traditional dense accelerators, and also delivers significant performance gains over the existing sparse accelerator.

MAFlow: A Memory-access Optimized CNN Inference Dataflow for Large-sized Feature

Due to the limited memory bandwidth of embedded systems, Convolutional Neural Networks (CNNs) often suffer from slow inference speeds when processing large-sized feature maps. This paper proposes a dataflow design named MAFlow, which includes the design of a hardware architecture and a five-step on-chip convolution pipeline. By eliminating repetitive off-chip data requests and greatly reducing memory suspension during convolution operations, MAFlow significantly enhances memory bandwidth utilization and reduces convolution latency. In the simulation for the convolution layers in AlexNet, MAFlow achieves a 16.5% reduction in DRAM access and a 72.6% reduction in latency at the cost of 1.69 MB of additional on-chip storage overhead. In the simulation for convolution operation with 2000×2000 input feature map, MAFlow maintains the DRAM bandwidth utilization rate above 84% and achieves a latency of 62.4 ms.These results highlight MAFlow's effectiveness in accelerating CNN inference for large-sized feature in memory-intensive applications.

Fb-FaceBoxes: FPGA-based face detection using repetitive kernels and memory-access optimization

Key Algorithms of Convolutional Neural Networks and Hardware Implementation of Image Processing

Edge computing and artificial intelligence have made the efficient deployment of machine vision algorithms on low-power hardware a critical challenge for integrated circuit design. Given data-intensive image pixels and deep neural network tensors, traditional von Neumann architectures inevitably encounter severe memory and power wall bottlenecks. This paper reviews low-level hardware acceleration strategies to deconstruct the mapping from algorithm logic to silicon substrates. Through systematic analysis, physical dimensionality reduction and operator substitution effectively reduce computational complexity and enhance energy efficiency. For visual preprocessing, line-buffer topologies and multiplier-less shift-and-add designs optimize bandwidth and eliminate DSP dependencies for certain edge detectors. In edge-based inference, 8-bit fixed-point quantization slashes system bus bandwidth demands by over 75% while restricting accuracy degradation to under 0.5%. Concurrently, Winograd fast convolution translates spatial convolutions into transform-domain mappings, substantially reducing core multiplier utilization and driving significant system throughput. Additionally, advanced dataflow scheduling circumvents power-hungry off-chip memory accesses by maximizing on-chip data reuse and resolving dynamic energy crises. Ultimately, these findings provide strong guidelines for the hardware-software co-design of emerging ultra-low power edge Artificial Intelligence (AI) chips.

Flash-DWC: Making Depthwise Convolution Compute-Efficient on GPUs

Depthwise convolution (DWC) is a key operator in efficient convolutional neural networks (CNNs). Recent models increasingly employ large-filter DWC to capture long-range dependencies, achieving accuracy competitive with that of vision transformers. Yet on GPUs, DWC remains far less efficient than its low FLOPs suggest and can become a runtime bottleneck, due to low arithmetic intensity, Tensor-Core mismatch, and inefficient memory access. To tackle these issues, we present Flash-DWC, a GPU library that makes DWC compute-efficient. Flash-DWC introduces a GEMV-based dataflow co-design that jointly optimizes data reuse, access order, layout, and workload tiling. This design increases arithmetic intensity, while preserving access contiguity and coalesced 128-byte memory accesses. Flash-DWC further transforms DWC GEMVs into compact high-density GEMMs on Tensor Cores, enabling efficient mixed-precision acceleration. In addition, matrix compression, a dual-path split-GEMM pipeline, and register-based input reuse further free on-chip resources, hide memory latency, and improve occupancy. We extend these techniques to both forward and backward propagation for efficient end-to-end training. Compared with the fastest cuDNN DWC algorithm, the GEMV-based CUDA-Core kernels and GEMM-based Tensor-Core kernels of Flash-DWC achieve 5.09 × and 9.01 × average speedups, respectively. In CNN training, Flash-DWC attains a 1.03 × –14.18 × speedup and matches PyTorch’s convergence on ImageNet-1K. By making DWC compute-efficient, Flash-DWC promotes the use of larger and more expressive filters.