#artificial intelligence
May 2026
PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving
PEEK maintains an incremental radix tree over the pending queue, exposing prefix-sharing clusters no existing engine surfaces, and matches the tree against the engine's prefix cache to yield longest-prefix-match for every waiting request.
Bing Xie, Zhi-Peng Wang, Masahiro Tanaka et al.
· arXiv.org · 0 citations