BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference
The key insight in BOOST is to use kernel access patterns to make page allocation and runtime data management wave-aware, and applies modulo-based page placement that eliminates access-ratio variance.
A. Saxena, J. Ju, Hritvik Taneja et al.
· 0 citations