MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models
This work proposes MOEMB, which instead scales UME along the expert axis through mixture-of-experts (MoE), growing encoder capacity while preserving single-vector, non-autoregressive encoding, and conducts the first comprehensive study of adaptive computation for MoE-based embedding.
Xuan-Ming Cui, S. Mishra, Wen-Tao Bao et al.
· 0 citations