Approximate Computing for Energy-Efficient AI Inference
TL;DR
For moderate levels of approximation, the energy saving was significant and larger for more aggressive approximation; for aggressive approximation with less precision, the drop in accuracy was steep, and the paper ends with a discussion on accuracy vs. energy.
Abstract
With the increasing computational and energy requirement for deep neural network (DNN) inference, energy efficiency is becoming a first-order system design constraint for deploying an AI system on power and battery constrained platforms. One attractive design paradigm that has recently emerged is approximate computing, which leverages the existing error resilience of DNNs to sacrifice a fraction of accuracy for a much bigger reduction of energy consumption. This research employed an experiment-based computer-systems evaluation approach to assess the effectiveness of approximate computing techniques in achieving energy efficiency for AI inference without compromising computational accuracy. Using selected deep learning models, a representative set of AI inference workloads, such as image classification workloads and workloads performing prediction using neural networks, were implemented, and various approximation techniques were introduced at different computational levels: reduced numerical precision, approximate arithmetic operations, quantization, and selective computation, with a conventional exact-computing implementation acting as the baseline against which the performance of the various approximations was compared. To demonstrate the reporting procedure this experimental design suggests, illustrative, not hardware-measured results are reported and interpreted as what these metrics would look like if hardware-energy measurements were made. The results presented in the illustrative pattern matched well the results which are abundantly reported in the approximate-computing literature: for moderate levels of approximation, the energy saving was significant and larger for more aggressive approximation; for aggressive approximation with less precision, the drop in accuracy was steep. The paper ends with a discussion on accuracy vs. energy, how this trade-off would affect the adoption of AI to constrained resources, and future directions for research validated by hardware.