Pagoda: Time and Energy Rooflines for DNN Workloads on Edge Accelerators Across Power Modes
TL;DR
This work develops time and energy roofline characterizations for Orin AGX across power modes, coupled with analytical models of compute and memory for DNN inference, and extends these models to DNN training, and demonstrates power-mode tuning that achieves up to \(15\%\) lower energy with small inference-time impact for some workloads.
Abstract
Edge accelerators such as Nvidia Jetsons are an integral part of the computing continuum, and are used for DNN inference and training. Jetson edge devices have 2000+ CUDA cores within a 70W power envelope and offer 1000s of power modes to customize CPU, GPU and memory frequencies, enabling diverse power–performance trade-offs for energy-constrained deployments. While data-driven methods exist to predict power and latency, there is limited principled understanding of why different power modes perform the way they do. We develop time and energy roofline characterizations for Orin AGX across power modes, coupled with analytical models of compute and memory for DNN inference. These reveal quantitative insights, e.g., the default power mode is not the most energy-efficient and time efficiency implies energy efficiency. We extend our models to DNN training, and demonstrate power-mode tuning that achieves up to \(15\%\) lower energy with small inference-time impact for some workloads.