GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving
GrowPage is introduced, an on-demand KV budgeting framework that treats KV capacity as a runtime resource and achieves a superior performance--throughput trade-off over existing approaches.
Qian-Kun Ma, Yan-Jiang Zhou, Zi-Nan Xiong et al.
· 0 citations