Dispelling the Cloud Mist: Predicting the Performance of Cloud LLMs with a Random Forest Method Fusing Multi-Dimensional Features

Abstract

With the explosive development of LLM-empowered agent technology, LLM inference performance has become more important than training. Cloud computing is a popular deployment approach, where performance prediction is vital for instance selection and QoS assurance. However, prediction is challenging due to GPU hardware heterogeneity, Transformer operator variations, and dynamic inference configurations. Virtualization and other features vary across clouds, further increasing prediction difficulty. Existing methods suffer from low accuracy and poor generalization. To tackle these issues, we propose Dispeller, a prediction model for GPU-accelerated cloud environments with three feature sets: 1) basic GPU hardware feature with 7 dimensions; 2) operator-level GPU performance feature with 4 dimensions; 3) inference configuration feature with 4 dimensions. We conduct experiments on public cloud GPUs and collect a real-world dataset of 10,112 samples. Random Forest is adopted to learn the nonlinear mapping between features and performance. Experimental results show that Dispeller achieves high prediction accuracy with TPS $\mathrm{R}^{{2}} = 0.951$ on seen GPUs and 0.989 on unseen GPUs, demonstrating strong cross-GPU generalization. An ablation study confirms that inference configuration features contribute 87.3% of the predictive power. Dispeller is therefore able to recommend cloud resources and optimize LLM deployment costs.