Hengshuai Yao

Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts Models

It is found that perplexity and downstream accuracy favor different MoE experts, cautioning against selecting MoE compression settings using unlabeled text alone, and showing that expert identity matters substantially more than expert weighting, while balanced and domain-specialized routing leaves limited room for expe...