Soonhoi Ha

MeanField Surrogate Modeling for Scalable Runtime Scheduling of Concurrent Heterogeneous AI Inference on Shared GPUs

A MeanField surrogate is proposed that predicts per-model performance from local configuration and aggregate GPU state rather than explicitly modeling all joint interactions, showing high predictive accuracy and empirical sample budget that grows approximately linearly in $N, in contrast to the combinatorial cost of fu...