Output-Aware Rotation for INT2 KV-Cache Quantization
The key-value (KV) cache has become a major memory and bandwidth bottleneck in long-context large language model inference, making ultra-low-bit quantization increasingly important. However, existing rotation-based INT2 methods optimize cache statistics or proxy errors before the complete attention readout, even though...
Vincent-Daniel Yun, Woo-Sang Lim, Minsoo Cheong et al.
· 1 citation