Ying-Wen Chen

Dispelling the Cloud Mist: Predicting the Performance of Cloud LLMs with a Random Forest Method Fusing Multi-Dimensional Features

With the explosive development of LLM-empowered agent technology, LLM inference performance has become more important than training. Cloud computing is a popular deployment approach, where performance prediction is vital for instance selection and QoS assurance. However, prediction is challenging due to GPU hardware he...