Bin Guo

TideKV: Delayed KV eviction via layer-wise chunked prefill for memory-efficient long-context LLM inference

TideKV, an on-GPU cache-construction architecture using layer-major delayed eviction that substantially expands feasible context lengths without sacrificing cache utility is presented.