Tool-augmented large language model (LLM) agents are becoming an important execution unit in service computing, but existing agent loops still lack explicit runtime signals for assessing task completion. The challenge lies in the fact that an agent may continue reasoning or invoking services even after the runtime cont...
Wan-Yi Zheng, Min-Xian Xu, Kan Hu et al.· 0 citations
C cumulative ablations show that adding the three mechanisms successively increases geometric-mean throughput, while budget adjustment improves accepted-token retention, and DScale is presented, preserving drafter architecture, weights, and full draft length.
Rong-Jian Chen, Min-Xian Xu, Zheng-Xin Fang et al.· 0 citations
An SSD-native local MoE inference system that addresses challenges with an SSD I/O pipeline optimized for expert delivery, a native three-tier storage hierarchy that delivers and retains experts dynamically, and balanced CPU--GPU hybrid execution.
Fang-Zhou Liang, Yibin Shen, Jian-Min Hu et al.· 0 citations
Inspired by the brownout paradigm in service computing, BrownoutMoE reorganizes experts into groups to improve utilization and system efficiency while maintaining service quality and introduces a grouping-consistent distillation process to produce deployable models that are compatible with standard inference pipelines.
Yi Ding, Minxian Xu, Zheng-Xin Fang et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
