Aug 2026
Learning to schedule: interference-aware optimization for edge AI inference on shared GPUs
A time-varying integer program to minimize the long-term total cost of the edge AI inference system, including the inference latency, the inference error rate, the query-dispatching communication cost, and the energy consumption, subject to resource and workload constraints is proposed.
Mingtao Ji, Hehan Zhao, Lei Jiao et al.
· Science China Information Sc... · 0 citations