Preprint
Aug 2026
Which Decisions Low-Bit Quantization Breaks, and How to Predict Them
This work tracks quantization across 16 models from 8 families under round-to-nearest, seven under AWQ, two under GPTQ and one under GGUF, at 8 down to 2 bits, and measures the margin, the picked option's score minus its best alternative's, which removes the protection a large margin affords.
Zekun Wu, Swati Dhiman, Adriano S. Koshiyama
· 1 citation