AdaOffload is presented, a general and efficient multi-GPU hybrid expert execution framework that effectively handles heterogeneous hardware and workloads and co-designs GPU-CPU parallelism and GPU-CPU co-execution for decode and prefill with online workload observation that adapts to different hardware, workload, and...
William X. Zheng, Zhong-Jie Chen, Shuo Yang et al.· 0 citations
Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granularity of kernels, on separate streams, reduces only part of this communication cost. Fused kernels often have better performance by transmitting each output tile as soon a...
Zi-Ming Mao, Yi-Han Zhang, S. W. Chew et al.· 1 citation
An assurance-revision loop that uses deployment evidence to revise the requirements, model, or evaluator when stakeholders reject the resulting behavior is proposed, which cast assured agentic development as a resource-allocation problem over human judgment, agent capability, and compute.
Alexander Krentsel, Shubham Agarwal, M. Cemri et al.· arXiv.org· 2 citations
Evaluating leading frontier and open-source code generation models on both intra-node NVLink and inter-node RDMA platforms reveals that even the strongest model, GPT-5.5, correctly implements and achieves competitive performance on only 30.7\% of the benchmark tasks.
Shuang Ma, Yu-Yi Li, Yihan Zhang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.