Rajalakshmi Srinivasaraghavan

Intelligent Inference Endpoint Scheduling for Heterogeneous Large Language Model Deployments: A Six-Dimensional Routing Framework

A framework is intended as a design reference for practitioners building inference gateways and as a structuring basis for future empirical work on multi-objective LLM request scheduling, and an evaluation protocol comprising workload definitions, baselines, and metrics is specified.