Flash-DWC: Making Depthwise Convolution Compute-Efficient on GPUs

Abstract

Depthwise convolution (DWC) is a key operator in efficient convolutional neural networks (CNNs). Recent models increasingly employ large-filter DWC to capture long-range dependencies, achieving accuracy competitive with that of vision transformers. Yet on GPUs, DWC remains far less efficient than its low FLOPs suggest and can become a runtime bottleneck, due to low arithmetic intensity, Tensor-Core mismatch, and inefficient memory access. To tackle these issues, we present Flash-DWC, a GPU library that makes DWC compute-efficient. Flash-DWC introduces a GEMV-based dataflow co-design that jointly optimizes data reuse, access order, layout, and workload tiling. This design increases arithmetic intensity, while preserving access contiguity and coalesced 128-byte memory accesses. Flash-DWC further transforms DWC GEMVs into compact high-density GEMMs on Tensor Cores, enabling efficient mixed-precision acceleration. In addition, matrix compression, a dual-path split-GEMM pipeline, and register-based input reuse further free on-chip resources, hide memory latency, and improve occupancy. We extend these techniques to both forward and backward propagation for efficient end-to-end training. Compared with the fastest cuDNN DWC algorithm, the GEMV-based CUDA-Core kernels and GEMM-based Tensor-Core kernels of Flash-DWC achieve 5.09 × and 9.01 × average speedups, respectively. In CNN training, Flash-DWC attains a 1.03 × –14.18 × speedup and matches PyTorch’s convergence on ImageNet-1K. By making DWC compute-efficient, Flash-DWC promotes the use of larger and more expressive filters.