Reinforcement learning (RL) post-training with generative reward models (GenRM) has become a central recipe for improving large language models (LLMs), and it is also increasingly resource-intensive: policy rollout, GenRM-based reward evaluation, and policy training all compete for GPU resources. However, existing colo...
Yan-Feng Zhao, Hao-Yang Li, Sheng Lin et al.· 0 citations
Sequential recommendation aims to predict the next item that aligns with user preferences based on their historical interaction sequence. Traditional sequential recommendation methods can be seen as understanding-based approaches, where the next relevant item is determined by analyzing user historical interactions and...
Yan-Wei Xu, Ye Tian, Fang-Cheng Fu et al.· IEEE Transactions on Knowled...· 0 citations
Mixture-of-Experts (MoE) has increasingly become a mainstream approach for scaling large language models, as it expands model capacity while keeping computation cost nearly constant. Training large-scale MoE models relies on Expert Parallelism (EP), which distributes expert replicas across GPUs and exchanges tokens thr...
Janus is a Service-Engine co-designed multi-model serving system built around dual-timescale scheduling that maintains 0.97-1.0 SLO attainment, versus 0.80-0.92 for the strongest baseline, while using 13,440 device-hours/day, 27% less than static Serverless-LLM.
Tian-Bao Zhou, Yi Wang, Yu Zhou et al.· Proceedings of the ACM SIGOP...· 2 citations
It is shown that pipeline parallelism (PP), long overlooked because it offers little decode-latency advantage, can reduce JCT by providing a more favorable balance between prefill and decode efficiency, and PipeSwift is built, an optimized open-source pipeline-parallel runtime integrated with a tailored micro-batch par...
Shiju Wang, Fei Ren, Fang-Cheng Fu et al.· arXiv.org· 0 citations
This paper proposes DPP, which transforms PP granularity from a static design choice into a workload-adaptive optimization space over packed, split, and hybrid chunks and further introduces a new coupling between heterogeneous pipeline scheduling and gradient checkpointing.
Shiju Wang, Yujie Wang, Fang-Cheng Fu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.