What Attention Recalls and Recurrence Controls in Hybrid Language Models
Two cache-level interventions are introduced: Split-prefill keeps only the KV cache or only the recurrent state from a prefilled context, then generates an answer, and state-swap pairs the KV cache from one context with the recurrent state from another in a single forward pass.
Kirill Afendulev, Alexey Dontsov, Elena Tutubalina et al.
· 2 citations
· ⚡1