KV-Kaizen: Learning Context-Adaptive Cache Compression Choices
This work learns a selector that is able to produce, based on context, a per-layer cache configuration towards an overall compression budget, and achieves the Pareto frontier of accuracy against cache size, against learning-free and post-hoc baselines.