Preprint
Aug 2026
DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding
DeaMoE is proposed, a decoding-efficient MoE architecture, in which the experts are grouped into several departments, and the experts belonging to the same department share most parameters since they come from the same professional field, and additionally each expert contains a few private parameters to reflect its uniqueness.
Zewen Jin, Shengyu Fu, Zeping Duan et al.
· 0 citations