CrystalLLM is presented to decouple large-scale execution from the need to access large clusters, enabling engineers to run and observe ranks of interest under faithful large-scale behavior using only a few GPUs.
Shao-Ke Xi, ChonLam Lao, Bo-Yi Jia et al.· Proceedings of the ACM SIGOP...· 0 citations
This work argues for co-design, that facilities should treat agents as first-class principals where agents become facility-aware tenants, and outlines the resulting challenges and opportunities in compute, storage, agent memory, and safety.
Yun-Jia Zheng, Bintang Dwi Marthen, Zachary Pan et al.· arXiv.org· 0 citations
This work studies production traces from two companies and evaluates 14 eviction algorithms across HBM-constrained and large memory-pool settings to show that effective prefix-cache management should retain recency as its foundation while selectively adding quick demotion for one-hit prefixes.