Portable to Efficient: Auto-Tuning Hardware-Agnostic GPU Kernels in Julia
TL;DR
The established Kernel Tuner auto-tuning framework is rebuilt with Julia support, enabling systematic exploration of kernel configurations for hardware-agnostic GPU kernels targeting NVIDIA, AMD, Intel, and Apple GPUs.
Abstract
Traditionally, GPU kernels have been developed and optimized within vendor-specific programming models to achieve high performance, resulting in software that is difficult to optimize and adapt across increasingly heterogeneous computing systems. Hardware-agnostic programming models offer a more sustainable approach to GPU software development by improving portability and maintainability, but achieving efficient execution across diverse architectures remains challenging. We address this challenge by integrating auto-tuning into hardware-agnostic GPU kernels written in Julia. We rebuild the established Kernel Tuner auto-tuning framework with Julia support, enabling systematic exploration of kernel configurations for hardware-agnostic GPU kernels targeting NVIDIA, AMD, Intel, and Apple GPUs. We demonstrate this approach on hardware-agnostic singular value decomposition (SVD) as implemented in the NextLA.jl linear algebra library. The results show that auto-tuning is essential for creating resource-efficient hardware-agnostic GPU kernels across a variety of hardware. Optimal configurations improve kernel performance by a factor of 3 × to 7 × compared to median parameter configurations, demonstrating the substantial impact of tuning on efficient hardware utilization. Moreover, auto-tuning enables hardware-agnostic kernels to surpass manually optimized implementations in efficiency while reducing developer effort. This is made widely accessible through the first auto-tuning framework for Julia, KernelTuner.jl, bringing portable yet resource-efficient GPU kernels within reach for a wide range of applications and devices.