HyQuant: Hybrid-Precision Quantization for LLM Attention
Across diverse tasks, models, and datasets, HyQuant maintains nearly lossless accuracy with an extremely simple design, demonstrating the efficiency and practical feasibility of hybrid quantization for LLM attention.
Jiarui Ding, Bing-Xin Xing, Yu Zhang et al.
· 0 citations