Jul 2026
Elastic Memory Remapping for Multi-tenant LLM Serving
This work presents Oneiros, a dynamic remapping engine for multi-tenant LLM serving that dynamically repurposes GPU memory allocated for model parameters as KV cache capacity, enabling nonblocking, unidirectional parameter transfer.
Ruihao Li, Shagnik Pal, Vineeth Narayan Pullu et al.
· ACM SIGOPS Operating Systems... · 0 citations