ARM: Attention with Routed-Memory for Learnable Sparse Control
Despite advances in long-context inference, large language models (LLMs) remain fundamentally limited by the key-value (KV) caching mechanisms that are necessary for stable computation. Techniques such as selective token eviction and pruning have vastly mitigated these issues, but often discard core information to mana...
Qiu-Hao Zeng, Jerry M. Huang, Peng Lu et al.
· 0 citations