Ye Yue

Plug-and-Play KV Cache Reuse for Low-Latency Retrieval-Augmented Generation

This work studies KV cache reuse for RAG under a strictly plug-and-play regime, where the base language model is left unchanged and no retraining, prompt redefinition, or selective recomputation is performed, and proposes a fine-tuning-free approach that reuses KV states across queries through an anchor-block and posit...