Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules
Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference requires increasingly large KV caches. Efficient recurrent alternatives reduce the asymptotic complexity of inference, but their sequential dependence limits token-wise p...
Sarah Pan, Ali Behrouz
· 0 citations