Skip to content

CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters

Sep 2026 · 0 citations · 28 references
Computer Science

TL;DR

CacheReforge, which represents stale KV caches as layerwise mixed-version objects, combines per-layer adapter anchors, calibrated sensitivity, accumulated drift, and executable restart boundaries to select direct reuse, bounded recomputation, or complete affected-suffix recovery.

Abstract

Large language models rely on KV caching to reduce repeated prefill computation in long context and interactive applications. As lightweight adapters evolve, cached states reflect earlier versions, so stale reuse distorts current model outputs, while complete affected suffix recomputation restores fidelity at substantial cost. We seek minimal recomputation that recovers current adapter behavior. Existing systems track token, context, or stable adapter identity, but neither represent caches from earlier adapter versions nor distinguish update propagation from the recomputation required for behavioral recovery. To address these gaps, we introduce CacheReforge, which represents stale KV caches as layerwise mixed-version objects. It combines per-layer adapter anchors, calibrated sensitivity, accumulated drift, and executable restart boundaries to select direct reuse, bounded recomputation, or complete affected-suffix recovery. We distinguish dependency depth from the functional recomputation horizon and use cumulative tail influence to characterize when bounded recovery preserves current-model behavior. We evaluate CacheReforge on Qwen2.5-1.5B and Qwen2.5-7B with continual LoRA updates, including 16K HotpotQA and 2WikiMQA workloads. CacheReforge reduces mean KL divergence by 92.4% relative to stale reuse, while recomputing only 5.44% of layers and reducing cache-maintenance time by 93.2% relative to fresh full prefill. These results show that version-aware recovery preserves model fidelity and most KV caching gains.

View source

Similar papers

ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation

Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encoding this content for every request wastes computation. Position-independent caching (PIC) alleviates this by encoding each artifact independently and reusing its key-value...

Xing-Hao Chen, Jun-Nan Dong, Cai Ke et al. · 0 citations
Book Open access Sep 2026

Disaggregated LLM KV-Cache Storage Requires Coordination-Free Consistency Abstractions

An available KV-cache hit is not always a safe decoder hit. A miss loses reuse; an invalid commit can silently perturb decoder output. In a disaggregated cache, a replica may know where a block was placed, but retirement, allocation reuse, and reader lifetimes evolve elsewhere. We use the CALM monotonicity criterion to...

Kalyan Pullela, H. B. Acharya · 0 citations
#machine learning Preprint Aug 2026

PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts

PatchKV, a profile-guided recovery system for suffix-preserving revisions, decomposes adjacent context versions into an exact prefix, an updated span, and an aligned suffix, and achieves a speedup in mean resume time-to-first-token over full suffix recomputation.

Guo-Tao Yang, Rui Guo, Si-Wei He et al. · 0 citations

When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse

This work studies production traces from two companies and evaluates 14 eviction algorithms across HBM-constrained and large memory-pool settings to show that effective prefix-cache management should retain recency as its foundation while selectively adding quick demotion for one-hit prefixes.

Yi-Yu Liu, Min-Lan Yu, Jun-Cheng Yang · 3 citations
#artificial intelligence Preprint Oct 2026

AvoKV-E: Payload-Aware KV Cache Eviction for Long Reasoning

AvoKV-E is introduced, a training-free eviction policy that first delays eligibility for recent states and then ranks eligible entries using candidate-normalized read pressure, key redundancy, and value-payload potential, which shows that long-reasoning KV eviction should preserve not only keys that are likely to be re...

Han Yu, Wen-Hui Zhu, Xi-Wen Chen et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.