Disaggregated memory architecture has gained wide adoption in cloud and high-performance systems [18, 31, 43] due to its decoupled resource model, elasticity, and low-latency access. In such architectures, transaction mechanisms must ensure atomic and consistent access to remote memory. Prior designs use array-based ve...
Ao-Xin Wei, Jin-Tian Wu, Jian Zhou et al.· Proceedings of the Internati...· 0 citations
Long-context LLM serving is increasingly bottlenecked by decode, where large KV caches limit batch size and underutilize GPUs. Sparse KV cache offloading expands effective capacity by storing most historical KV blocks in CPU DRAM and recalling only selected blocks on demand. However, we find that existing offloading sy...
Qiu-Yang Zhang, Kai Zhou, Kai Lu et al.· 0 citations
Disaggregated memory architecture has gained wide adoption in cloud and high-performance systems [18, 31, 43] due to its decoupled resource model, elasticity, and low-latency access. In such architectures, transaction mechanisms must ensure atomic and consistent access to remote memory. Prior designs use array-based ve...
Ao-Xin Wei, Jin-Tian Wu, Jian Zhou et al.· Proceedings of the Internati...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
