Plug-and-Play KV Cache Reuse for Low-Latency Retrieval-Augmented Generation
This work studies KV cache reuse for RAG under a strictly plug-and-play regime, where the base language model is left unchanged and no retraining, prompt redefinition, or selective recomputation is performed, and proposes a fine-tuning-free approach that reuses KV states across queries through an anchor-block and posit...
Ye Yue, Zhe-Wei Wang, Fabio Agosto et al.
· 0 citations