FidelityQ: Preserving Predictive Fidelity in Ultra-Low-Bit Language Model Quantization
Ultra-low-bit scalar quantization offers a regular arithmetic structure for efficient deployment of large language models, but preserving predictive behavior at extreme precision remains challenging: with only a few reconstruction levels, accurate weight representation must translate into consistent predictions from a...
Yu-Jie Chen, Wei-Yan Zheng, Yi-Wu Yao et al.
· 0 citations