PRQuant: Permutation Residual Quantization for Low-Overhead Inference
Low-bit quantization of linear layers is often dominated by a small number of outlier channels. Existing smoothing, rotation, and residual-based methods can mitigate this issue, but may shift the quantization bottleneck to weights or introduce costly online operations. To address these limitations, we propose PRQuant (...
Pei-Ran Wang, An-Qi Wang, Jia-Ying Zhao et al.
· 0 citations