Latent Compaction with Test-Time Training
Long contexts enlarge key--value (KV) caches, increasing memory use and attention costs during generation. Compressing these caches must preserve information for unseen questions. We introduce **Latent Compaction**, which learns a compact document-specific cache through test-time training. We initialize cache entries u...
Yingfa Chen, Xing-Yu Shen, Chao-Jun Xiao et al.
· 0 citations