Why Does Post-Training Quantization Work?
Post-training quantization compresses large language models (LLMs) by storing their weights at reduced precision, and each quantized weight introduces an error into the hidden states. Naively, these errors should accumulate with depth and corrupt next-token prediction; randomly initialized models accumulate these discr...
Yu-Xiang Chen, Michael Beyer, Jun Zhu et al.
· 0 citations