H. J. Chao

Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inference

Power availability increasingly constrains the operation of AI inference fleets, creating a need for provisioning methods that jointly consider serving capacity and power consumption. Prefill--decode (PD) disaggregation has emerged as a prevalent architecture for large-scale inference serving. However, determining the...