Long-context inference makes the key-value (KV) cache a major memory bottleneck for large language models. We propose BELT, a low-rank KV cache compression framework. Its key-side objective targets attention distributions rather than raw keys or pre-softmax logits, yielding a quadratic surrogate with a closed-form low-...
Yao-Yang Liu, Shu-Wei Shen, Jin-Cheng Tang et al.· 1 citation
OnPTQ is proposed, an on-policy framework that calibrates on trajectories visited by the current quantized policy, and derives a Decision--Consequence bound linking behavioral deviation to current policy discrepancy and action-conditioned future-value span.
Wen-Xiao Fan, Jing-Ling Fu, Li-Chen Ma et al.· 0 citations
We present Hunyuan-A13B, an open-source large language model based on a Mixture-of-Experts architecture. It contains 80 billion total parameters but activates only 13 billion during inference, balancing model capability, computational efficiency, and deployment cost. The model is pretrained on a rigorously filtered 20T...
Tencent Hunyuan Team, Ao Liu, Bo Zhou et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.