ScoutRoute: Efficient Mixture-of-Experts Routing Using Internal Language Model Signals
Efficient expert routing is a fundamental challenge in Mixture-of-Experts (MoE) systems, particularly for heterogeneous collections of independently trained Large Language Mod els (LLMs). Existing routing approaches typically rely on jointly trained gating networks, prompt-based predictors, or computationally expensive...
Rajdeep Pandhare, Delzaan Bharucha
· 0 citations