Chaitra Samant

To Evict or to Merge? A Survey of LLM KV Cache Management Strategies and the SR-KV Framework

Large Language Models (LLMs) rely on a Key-Value (KV) Cache to avoid recomputation of attention information for the previously generated tokens, but the cache keeps growing linearly with the sequence length and thus quickly becomes the dominant memory bottleneck during long context inference. This paper presents a surv...