AIMER: Calibration-Free Task-Agnostic MoE Expert Pruning
This work proposes AIMER (AIMER), a simple calibration-free criterion that identifies more distinct experts by capturing the concentration pattern of expert weights, making it well suited for task-agnostic expert pruning.
Zong-Fang Liu, Guang-Yi Chen, Yi-Fan Shen et al.
· Conference on Empirical Meth... · 7 citations