HeadWiseKV: Budgeted Per-Head Cache Residency for Hybrid Long-Context Language Models
This work introduces HeadWiseKV, a training-free framework that compresses the residual global KV caches of hybrid language models while preserving their native local, recurrent, and linear paths and proposes SeqCalib as the core policy-generation algorithm in HeadWiseKV.
Ren-Jie Xie, Jun-Cheng Yang, Ao-Ting Hu et al.
· 0 citations