Mixture-of-experts (MoE) layers increase model capacity without a proportional increase in per-example computation. However, conventional flat routers can yield imbalanced expert utilization and treat experts as an unstructured collection, whose indices carry no topological meaning. We introduce {\bf BRANCH-MoE}, a rou...
Gang Fu, Adel Javanmard, M. Bateni et al.· 0 citations
The present work builds on the differential-equation method, itself a limiting form of the auxiliary-receiver approach in network information theory using a continuum of degraded receivers, and gives a computer-assisted proof of the Courtade--Kumar conjecture.
Zi-Jie Chen, Amin Gohari, Adel Javanmard et al.· 1 citation
This work proposes Subsampled Stochastic TurboQuant (SSTQ), a framework that combines overcomplete equal-norm tight frames, coordinate subsampling, and privacy-aware one-dimensional quantization and empirically evaluates SSTQ against established baselines on federated learning tasks using CIFAR-10 and Fashion-MNIST, de...
Adel Javanmard, David P. Woodruff, V. Mirrokni· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.