This work presents MeshRT, the first wafer-scale system to achieve both ultra-low latency and high throughput for LLM inference via a novel system architecture, which is called compile-time governance of runtime dynamism.
Cong-Jie He, Le Xu, Zhan Lu et al.· Proceedings of the ACM SIGOP...· 0 citations
Wavel is presented, a fast and efficient compilation system for wafer-scale accelerators that introduces MeshIR to make executable schedules explicit, constructs them through two legal locality-preserving primitives, CUT and TUNE, and uses physical cost evaluation with structured pruning to keep search practical.
Ye-Qi Huang, Cong-Jie He, Hao-Cheng Xiao et al.· Proceedings of the ACM SIGOP...· 0 citations
This work proposes AgentSpec, a speculative decoding algorithm that addresses the limitations of existing methods for LLM agents and incorporates structure-isolated drafting that constrains speculation to semantically coherent segments of the agent workflow, reducing the drafts of irrelevant semantic paths and achievin...
Xin Wang, Zi-Ming Miao, Yi Zhu et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.