Jul 2026
Recall Before You Rank: Similarity-Guided Top-K Reuse for Efficient Long-Context Attention
ReTopK is a training-free method that accelerates dynamic Top-$K$ attention by reusing historical retrieval decisions and retains the complete KV cache and reuses only selected indices, rather than historical scores, attention weights, or outputs.
Wen-Shuai Yao, Wen-Yong Zhou, Hanyong Shao et al.
· arXiv.org · 0 citations