Preprint
Aug 2026
LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Serving
LLMVisor is presented, a roofline-guided latency attribution model that captures the memory-bound and compute-bound phases via a concise piecewise-linear form over features proportional to FLOPs and memory I/O traffic and runs efficiently at microsecond scale.
Shuowei Jin, Xueshen Liu, Jiaxin Shan et al.
· 2 citations