This work presents ToolSched, a Shell MCP Server for shared local hosts that serves multiple agents and schedules their operations, which improves resource utilization by 148.4% and end-to-end command throughput by 29.4%, demonstrating its effectiveness and practicality.
Abstract
Local LLM agents often work on shared hosts. Unlike multi-tenant cloud clusters, these hosts often have tightly bounded memory, CPU, and I/O resources. Concurrent agents frequently suffer from resource contention, leading to failures such as OOM and reduced resource efficiency. Considering that agents'resource demands are bursty and shell command delay is imperceptible to agents, there is a huge scheduling opportunity to arrange when and in what order commands are actually executed. Thus, we present ToolSched, a Shell MCP Server for shared local hosts that serves multiple agents and schedules their operations. ToolSched embeds a resource model into the MCP interface: agents estimate resource demands along with command requests. These hints, together with real-time host resource conditions, determine when commands are actually executed. Internally, ToolSched applies a scheduling algorithm to stagger commands, reducing contention and improving resource utilization. Prediction correction and checkpoint-based recovery mitigate estimation bias and resource errors. Our evaluation on Terminal Bench shows that, compared with unscheduled execution, ToolSched improves resource utilization by 148.4% and end-to-end command throughput by 29.4%, demonstrating its effectiveness and practicality.
The latency with respect to the resource dynamics of processing multiple requests and tasks concurrently is characterized and new optimization opportunities that exploit the resource dynamics of tasks are demonstrated: CPU-aware tool admission and task-aware CPU allocation.
Wonmi Choi, Minuk Park, Zhi-Xiong Niu et al.· arXiv.org· 1 citation
LLM agents execute as long-lived, multi-stage workflows with substantial resource-demand variation across phases. Provisioning each agent or sandbox for its lifetime peak leads to low resource utilization, particularly for memory, whereas aggressive oversubscription risks memory contention, out-of-memory failures, and...
Yi-Fan Zhang, Xin-Kui Zhao, Rui-Xin Huang et al.· 0 citations
TAO is presented, a tool-aware joint scheduling framework for LLM-based agentic serving that combines near-term tool readiness, node-specific cache credit with time-aware decay, and token footprint; a dynamic program jointly selects and places waiting sessions under capacity constraints.
Shu-Hua Li, Sheng-Dong Ye, Dong-Dong Ge et al.· 1 citation
Large language model (LLM)-based multi-agent systems (MAS) coordinate specialized agents to solve complex tasks, turning each user request into interdependent model and tool operations. Unlike independent requests, these operations become ready at different times and consume evolving context as agents coordinate and ex...
Sheng-Xuan Qiu, Hao-Chen Huang, Shu-Zhang Zhong et al.· 0 citations
ML-based resource managers—learned autoscalers, utilization-driven schedulers, RL-based controllers—were designed for workloads with stationary resource profiles. AI agents executing tool calls via the Model Context Protocol (MCP) violate every assumption these systems rely on: resource demand is non-stationary within...
Sheetal Sriram· Proceedings of the 5th Works...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.