MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models
This work proposes MOEMB, which instead scales UME along the expert axis through mixture-of-experts (MoE), growing encoder capacity while preserving single-vector, non-autoregressive encoding, and conducts the first comprehensive study of adaptive computation for MoE-based embedding.