Skip to content

RelaxKV: Recomputation Guided by the Query with Sparse Context Attention for Efficient KV Cache Reuse

Sep 2026 · arXiv.org · Vol abs/2609.33503 · 2 citations · 23 references
Computer Science

Abstract

Cross-request KV caching reduces the prefill cost of Retrieval-Augmented Generation (RAG), but conventional prefix caching severely limits cache reuse across requests. Position-Independent Caching (PIC) removes this constraint by reusing independent chunks, but their KV states miss cross-chunk interactions. Existing methods selectively recompute token states to recover these missing interactions, but primarily allocate the recomputation budget to selecting which states to recompute, while fixing the recomputation context to the full causal prefix. We introduce RelaxKV, which formulates selective cache repair as a joint allocation problem over repair targets and recomputation context. Guided by the user query, RelaxKV identifies layer-specific repair targets and restricts their recomputation to a query-relevant context, reducing attention computation. Across four decoder models, RelaxKV at a 15% anchor ratio improves aggregate LongBench performance over ProphetKV on all models. On Qwen3-14B, RelaxKV provides a stronger quality-TTFT trade-off than ProphetKV across a 5%-30% anchor-ratio sweep, and achieves the best selective results on RULER-MV and LV-Eval at 16K and 32K context lengths. Controlled ablations further demonstrate the importance of recomputation context selection.

View source

Similar papers

ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation

Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encoding this content for every request wastes computation. Position-independent caching (PIC) alleviates this by encoding each artifact independently and reusing its key-value...

Xing-Hao Chen, Jun-Nan Dong, Cai Ke et al. · 0 citations
Conference Open access Jun 2026

Plug-and-Play KV Cache Reuse for Low-Latency Retrieval-Augmented Generation

This work studies KV cache reuse for RAG under a strictly plug-and-play regime, where the base language model is left unchanged and no retraining, prompt redefinition, or selective recomputation is performed, and proposes a fine-tuning-free approach that reuses KV states across queries through an anchor-block and posit...

Ye Yue, Zhe-Wei Wang, Fabio Agosto et al. · 0 citations
Aug 2026

QV-PIC: Query-Aware Visual Position-Independent Caching for Efficient RAG Serving

QV-PIC is proposed, a query-aware dual-resolution PIC reuse framework guided by model-native templates that preserves global context with low resolution and restores fine-grained textual evidence within a high-resolution budget by cumulative query relevance scores, retaining the efficiency benefit of visual compression...

Yi-Lin Liu, Rui Meng, Wang-Ze Ni et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.