Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference
A Context-to-Answer-Aligned Memory Compression (CMC) framework is proposed, which compresses long input contexts into compact Context Memory Embeddings aligned to any frozen decoder's embedding space, reducing inference costs without modifying decoder weights.
Md. Mostafizer Rahman, Md. Faizul Ibne Amin, Md. Shahajada Mia et al.
· 1 citation