HBQ: Hierarchical Scaling Block Quantization with Hardware-Efficiency-Aware Design for Accurate LLM Inference
Hierarchical Block Quantization (HBQ) uses large blocks to maximize efficiency and introduces low-overhead significand (SIG) scaling for second-level quantization and compensates for large-block errors more effectively than prior PoT and INT schemes.
Chun-Ting Chen, Dongguang Han, Han-Gyeol Mun et al.
· 0 citations