Si-Wei He

PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts

Long-running LLM agent workflows often revise interior context spans while retaining long suffixes. Although suffix tokens remain unchanged, altered causal histories and rotary positions prevent exact reuse of their offloaded key-value (KV) states. Full suffix recomputation wastes prefill work, while indiscriminate reu...