LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference
LeanStream progressively refines computation, loading, and cache-retention priorities using partial GPU results, enabling fine-grained overlap between GPU execution and storage I/O.
Ren-Yuan Liu, Yu-Yang Leng, Kai-Yan Liu et al.
· 0 citations