Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration for Efficient Multimodal LLM Inference
Pro-Router is proposed, a token-aware progressive model routing method with adaptive edge-cloud collaboration for efficient multimodal LLM inference and achieves the highest routing accuracy and improves routing speed by more than 10x.
Xin-Yuan Gui, Shao-Wen Wang, Sheng Sun et al.
· 0 citations