ACache is presented, a cross-request cache reuse mechanism for shared spans, or affixes, at any position: prefix, infix, or suffix, and co-designs it with a modern inference engine, whose attention reads each request's recomputed Anchor KV states alongside one affix cache shared across concurrent requests.
Kai-Huan Liang, An Zhong, Xin Tan et al.· 0 citations
Libra is presented, an adaptive runtime for agentic RL post-training with two complementary components: intra-stage scheduling via a Causality-Guided Bucket Scheduler that routes requests across execution buckets with different parallelism configurations, reducing delays from rollout stragglers; and cross-stage coordin...
Kai Chen, Xin Tan, Jingzong Li et al.· arXiv.org· 3 citations· ⚡1
This work builds Silverstone to orchestrate dynamically both compute and network in disaggregated RL, using a reconfigurable optical-electrical fabric called RFabric that achieves superior performance-cost efficiency at scale over static Fat-Tree networks.
Xin Tan, Yicheng Feng, Yu Zhou et al.· Conference on Applications,...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.