Activation Sensitivity as a Unifying Principle for Post-Training Quantization

Abstract

Post-training quantization (PTQ) methods for large language models compress pretrained weights to low bit-widths without retraining by optimizing layer-local proxy objectives. Activation-aware scaling methods such as AWQ protect input channels with large activation norms, whereas second-order error-compensation methods such as GPTQ redistribute rounding error using the inverse of the unweighted input covariance matrix. We show that both paradigms -- along with diagonal Fisher quantization, block reconstruction, and Hadamard rotation methods -- are special cases of minimizing a single loss-derived quadratic form governed by gradient-weighted activation sensitivity. From a Gauss-Newton expansion of the task loss at a linear layer $Y = X W^\top$, we prove that under isotropic output curvature the expected excess loss from weight perturbation $\Delta W$ reduces to $\frac{1}{2 d_{\mathrm{out}}}\operatorname{Tr}(\Delta W H_D \Delta W^\top)$, where $H_D = \frac{1}{n} X^\top D X$ weights each calibration token's outer product by its squared output-gradient norm $D_{tt} = \|\nabla_{y_t}\mathcal{L}\|_2^2$. Within this framework, AWQ corresponds to a diagonal approximation of $H_D$ with uniform token weights ($D = I_n$) optimized via channel scaling, while GPTQ corresponds to Cholesky error compensation on $H_D$ under $D = I_n$. We derive closed-form optimal scaling exponents for diagonal sensitivity protection, formulate sensitivity-weighted Cholesky quantization (SAQ-GPTQ) on $H_D$, and prove bounds on token-gradient heterogeneity bias, cross-layer error amplification across residual blocks, and finite-sample calibration concentration. Controlled numerical simulations confirm that under token-gradient heterogeneity, replacing $X^\top X$ with $X^\top D X$ preserves $>99\%$ top-channel saliency recall and reduces INT3/INT4 excess loss by roughly $3\times$ over unweighted GPTQ.