MServerSim: A Modular Design based Simulation Technique for Modern AI Servers

Abstract

AI servers are the core infrastructure of current artificial intelligence computing, but their key components, such as computing power, storage, and interconnection, update rapidly. The growing component diversity and system complexity pose great challenges to AI server design, optimization, and performance evaluation. Currently, modeling and simulation are effective and widely used ways to reduce design costs and shorten R&D cycles of computer architecture. However, existing tools suffer from limited simulation capability, difficult cross-module integration, and poor adaptability to heterogeneous environments, which prevent them from performing system-level evaluation for modern AI servers. This paper proposes MServerSim, a modular design based simulation technique to address the limitations of the existing studies. Through a standardized framework and flexible adapters, MServerSim is able to integrate heterogeneous simulators with different fidelity levels and capabilities into a unified environment, enabling the simulation of AI server hardware, topology, and workloads, as well as supporting system-level evaluation. In this paper, we implemented a prototype of MServerSim, and adopted LLaMA2 training and DeepSeek-V3 inference as workloads for a simulated AI server, aiming to verify the modularization capabilities of MServerSim for workload, architecture, communication, component, and simulation model. We hope our work can support scenarios such as design optimization, module replacement, and bottleneck localization, helping accelerate the iteration of modern AI servers.