Large-scale agentic training and evaluation with large language models (LLMs) rely on isolated, stateful execution environments in which models inspect repositories, invoke tools, execute commands, and interact with task-specific services. These workloads create sandboxes in large bursts, span heterogeneous functionali...
Jia-Liang Huang, Hong-Xuan Tang, Jing-Chang Chen et al.· 3 citations
DSpark is introduced, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification, and enables performance tiers that were previously unattainable, shifting the Pareto frontier of the DeepSeek-V4 serving system.
DualPath is an inference system that breaks this bottleneck by introducing dual-path KV-Cache loading and enables a novel storage-to-decode path, in which the KV-Cache is loaded into decoding engines and then efficiently transferred to prefill engines via RDMA over the compute network.
Yong-Tong Wu, Shao-Yuan Chen, Rilin Huang et al.· Conference on Applications,...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
