Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference
PISA, a prompt-adaptive, cost-aware single-model router, and PISA-E, its extension to ensemble routing, are introduced and the effectiveness of prompt-adaptive capability estimation for both single-model routing and selective ensembling is demonstrated.
Saul Urso, Emanuele Carlini, Patrizio Dazzi et al.
· 0 citations