AdaOffload is presented, a general and efficient multi-GPU hybrid expert execution framework that effectively handles heterogeneous hardware and workloads and co-designs GPU-CPU parallelism and GPU-CPU co-execution for decode and prefill with online workload observation that adapts to different hardware, workload, and...
William X. Zheng, Zhong-Jie Chen, Shuo Yang et al.· 0 citations
This work instantiates VDN on MiniMax H3, which combines local Softmax attention with bidirectional linear memory for long-range video context and introduces Video Delta Attention (VDA), which updates memory once per frame by jointly incorporating its spatial tokens.
Frontier open-weight models are increasingly available, but serving them still largely assumes datacenter infrastructure. We present FreeToken, an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform. FreeToken co-designs the full serving stack,...
Shuo Yang, Xiao-yun Fan, Melissa Z. Pan et al.· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.