A novel quantization and accelerator design harnessing adaptive-precision (HAP), leverages a per-group dynamic zero-point to generalize APQ and achieves superior accuracy compared with existing PTQ solutions at the level of 4 or 5 bits for a variety of DNN families.
Abstract
Post-training quantization (PTQ) avoids expensive training processes and is effective in accelerating deep neural networks (DNNs). Since the representation capacity of a quantizer decreases exponentially with precision, PTQ encounters fundamental limitations in the low-precision domain. As an alternative, adaptive-precision quantization (APQ) leverages the coding redundancy in binary representations to reduce precision without sacrificing resolution. This is especially suitable for activations, as they exhibit greater exploitable redundancy and lesser robustness than weights in PTQ. However, existing APQ approaches require data to heavily cluster near zero, which is not guaranteed in asymmetric quantization that is preferred for activations. Moreover, the precision variability in APQ leads to an unbalanced computational workload, making it difficult to effectively harvest the theoretical performance gain. To address these challenges, a novel quantization and accelerator design harnessing adaptive-precision (HAP), leverages a per-group dynamic zero-point to generalize APQ. Its intra-channel grouping strategy further enables balancing workloads through reordering. Leveraging a dual-precision vulnerable channel protection scheme for weights, it achieves superior accuracy compared with existing PTQ solutions at the level of 4 or 5 bits for a variety of DNN families. A novel bit-serial accelerator features a lightweight reorder engine, achieving a 2.65× speedup and a 55% energy reduction on average compared to existing designs.
A new approach, called Q-PACE, consisting of a second-order sensitivity model that predicts the loss increase as a sum of quantization noise MSE weighted by per-layer curvature coefficients, and its stability during training allows for infrequent, cheap recalibration.
Alexandra Volkova, Matin Ansaripour, Erik Schultheis et al.· 0 citations
SSDi8 is presented, the first post-training quantization framework specifically designed for SSD to maintain a persistent INT8 path, and introduces a reformulation that decouples element-wise multiplications from matrix multiplications, enabling reuse of quantized activations across modules.
Hyunwoo J. Kim, Byoungchan Ko, Minseok Kang et al.· 3 citations· ⚡1
TR-PTQ, a unified integer-only formulation using shared Taylor Region exponential and logarithm primitives, is introduced, which eliminates the need for floating-point hardware units for nonlinearities and achieves less than 1.5\% absolute accuracy degradation across vision and language benchmarks.
Eli S. Levy, Adam Teman, Yoni Pugachov· 0 citations
APT, a software-hardware co-designed accelerator for high-resolution DiTs that leverages attention probabilities as a unified importance metric to jointly optimize computation through fine-grained pruning and adaptive precision scaling, and is evaluated on SOTA DiT models including PixArt, Stable Diffusion 3, and FLUX.
Sungyeob Yoo, Seeyeon Kim, Joonyong Park et al.· 0 citations
A new LLM compression method that co-optimizes pruning and quantization in a unified framework that performs favorably against two-stage baselines, even when subjected to extreme quantization settings designed for ultra-efficiency.