The major workloads in modern large language model (LLM) serving systems have shifted from single-shot LLM calls to multi-turn conversations, where new responses are generated based on the whole conversation history across all previous turns. The hit ratio, i.e., the average fraction of KV caches accessed directly from...
He-Yuan Yao, Chu-Tong Gao, Yuan Lyu et al.· 0 citations
This work introduces Bayesian in-context experimenters: transformer policies trained to imitate a Bayesian posterior Neyman teacher, and proves this amortized policy can be learned via empirical risk minimization using supervised pretraining.
Jiachun Li, David Simchi-Levi· arXiv.org· 0 citations
A contextual restless multi-armed bandit (CRMAB) framework in which a grid operator requests load reductions without observing internal job-scheduling decisions is proposed, demonstrating the economic potential of data-center flexibility as a grid service and highlighting the importance of high-quality, open-source AI...
Zi-Xi Chen, Yifu Ding, Rui-Cheng Ao et al.· 0 citations
This work develops an assumption-lean partial identification framework that uses such measurements as weak shadow variables, defined as outcome-informative proxies that are conditionally independent of missingness given the true outcome and observed covariates.
Hongyu Chen, David Simchi-Levi, Ruoxuan Xiong· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.