This work studies KV cache reuse for RAG under a strictly plug-and-play regime, where the base language model is left unchanged and no retraining, prompt redefinition, or selective recomputation is performed, and proposes a fine-tuning-free approach that reuses KV states across queries through an anchor-block and position- adjustment strategy.
A combined approach is proposed that fine-tunes the model while taking KV cache concatenation into account and selectively recomputes a subset of the KV caches and demonstrates improved accuracy for long-context inputs.
F. Tachibana, Daisuke Miyashita, Jun Deguchi· 0 citations
Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encoding this content for every request wastes computation. Position-independent caching (PIC) alleviates this by encoding each artifact independently and reusing its key-value...
Xing-Hao Chen, Jun-Nan Dong, Cai Ke et al.· 0 citations
Results show that separating reusable schema encoding from selective resource access substantially reduces agentic inference costs with limited effectiveness loss.
Yichu Fang, Si-Tong Wei, Hao-Zhe Hu et al.· 3 citations
Cross-request KV caching reduces the prefill cost of Retrieval-Augmented Generation (RAG), but conventional prefix caching severely limits cache reuse across requests. Position-Independent Caching (PIC) removes this constraint by reusing independent chunks, but their KV states miss cross-chunk interactions. Existing me...
Ruo-Ling Qi, Yi-Rui Liu, Xuan'er Wu et al.· 0 citations
Minima-KV, a retention-preserving hierarchy for mixed-format paged attention, is presented, a practical mixed-format path for compressing long-context state without evicting live-request KV pages.
S. Kozyrev, AI DavydMaiborodaMinima, Inc.· 0 citations
Context caching is widely adopted to reduce the dominant prefill cost for long prompts across various large language model (LLM) applications. Recently, position-independent caching (PIC) has emerged to improve KV cache reuse beyond strict prefix matching. However, the required KV caches often reside in a persistent re...
Jie-Shou Xu, Ren-Jie Liu, Yi Li et al.· Proceedings of the ACM on Ma...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
