To Evict or to Merge? A Survey of LLM KV Cache Management Strategies and the SR-KV Framework

Abstract

Large Language Models (LLMs) rely on a Key-Value (KV) Cache to avoid recomputation of attention information for the previously generated tokens, but the cache keeps growing linearly with the sequence length and thus quickly becomes the dominant memory bottleneck during long context inference. This paper presents a survey of training free strategies proposed to manage this bottleneck, organising them into four families: hard token eviction, cache merging and semantic compression, low rank reduction and quantization, and adaptive budget allocation across layers and heads. Across the reviewed literature, eviction based methods bound memory efficiently but discard context irreversibly, while merging based methods preserve information at the cost of disrupting the positional structure that Rotary Positional Embeddings (RoPE) depend on. Few existing frameworks address both limitations simultaneously. Motivated by this gap, the survey proposes SR-KV, a caching framework that combines a lightweight multi-signal importance score, centroid based merging of low importance tokens and a systematic evaluation of RoPE assignment strategies of merged tokens. The design, its constituent components and its positioning relative to existing approaches are described in detail; the reviewed literature is used to justify these design choices and to outline the experimental validation required before SR-KV’s benefits can be empirically confirmed.