Nexus is presented, a quasi-user-space I/O stack with a split architecture: a user-space data plane and a lightweight kernel control plane that enables efficient memory usage through shared data structures and in-kernel page reclamation that coordinate global allocation and reclamation across all processes.
Riwei Pan, Yu Liang, Sam H. Noh et al.· Proceedings of the 18th ACM...· 0 citations
HyMCache is presented, a KV-cache framework that integrates CXL-hybrid memory (CXL-HM) for multi-turn LLM serving that uses request-level prefix prefetching and opportunistic write buffering to stage latency-critical reads in device DRAM, enabling DRAM-scale KV-cache efficiency at SSD-level cost.
Hakbeom Jang, Inho Song, Sam H. Noh et al.· arXiv.org· 2 citations· ⚡1
This paper presents HyMCache, a CXL memory rack for multi-turn LLM serving using cost-efficient CXL-hybrid memory, which combines a small amount of in-device DRAM with large SSD-backed capacity behind a CXL interface to efficiently support TB-scale SSD-backed KV reuse.
Hakbeom Jang, Inho Song, Hoshik Kim et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.