Skip to content
Preprint

ToolSched: Resource-Efficient Tool Scheduling for Shared-Host LLM Agents

Nov 2026 · 0 citations · 27 references
Computer Science

TL;DR

This work presents ToolSched, a Shell MCP Server for shared local hosts that serves multiple agents and schedules their operations, which improves resource utilization by 148.4% and end-to-end command throughput by 29.4%, demonstrating its effectiveness and practicality.

Abstract

Local LLM agents often work on shared hosts. Unlike multi-tenant cloud clusters, these hosts often have tightly bounded memory, CPU, and I/O resources. Concurrent agents frequently suffer from resource contention, leading to failures such as OOM and reduced resource efficiency. Considering that agents'resource demands are bursty and shell command delay is imperceptible to agents, there is a huge scheduling opportunity to arrange when and in what order commands are actually executed. Thus, we present ToolSched, a Shell MCP Server for shared local hosts that serves multiple agents and schedules their operations. ToolSched embeds a resource model into the MCP interface: agents estimate resource demands along with command requests. These hints, together with real-time host resource conditions, determine when commands are actually executed. Internally, ToolSched applies a scheduling algorithm to stagger commands, reducing contention and improving resource utilization. Prediction correction and checkpoint-based recovery mitigate estimation bias and resource errors. Our evaluation on Terminal Bench shows that, compared with unscheduled execution, ToolSched improves resource utilization by 148.4% and end-to-end command throughput by 29.4%, demonstrating its effectiveness and practicality.

View source

Similar papers

Not All AI Agents Are Equal: Characterizing Resource and Performance Dynamics

The latency with respect to the resource dynamics of processing multiple requests and tasks concurrently is characterized and new optimization opportunities that exploit the resource dynamics of tasks are demonstrated: CPU-aware tool admission and task-aware CPU allocation.

Wonmi Choi, Minuk Park, Zhi-Xiong Niu et al. · 1 citation
Preprint Oct 2026

TenonAgent: Virtualizing Execution Residency for Memory-Oversubscribed LLM Agents

LLM agents execute as long-lived, multi-stage workflows with substantial resource-demand variation across phases. Provisioning each agent or sandbox for its lifetime peak leads to low resource utilization, particularly for memory, whereas aggressive oversubscription risks memory contention, out-of-memory failures, and...

Yi-Fan Zhang, Xin-Kui Zhao, Rui-Xin Huang et al. · 0 citations
Preprint Oct 2026

TAO: Tool-Aware Joint Scheduling for LLM-Based Agentic Serving

TAO is presented, a tool-aware joint scheduling framework for LLM-based agentic serving that combines near-term tool readiness, node-specific cache credit with time-aware decay, and token footprint; a dynamic program jointly selects and places waiting sessions under capacity constraints.

Shu-Hua Li, Sheng-Dong Ye, Dong-Dong Ge et al. · 1 citation
Preprint Oct 2026

MAServeBench: Benchmarking Multi-Agent LLM Serving Systems with Hierarchical Collaboration Workloads

Large language model (LLM)-based multi-agent systems (MAS) coordinate specialized agents to solve complex tasks, turning each user request into interdependent model and tool operations. Unlike independent requests, these operations become ready at different times and consume evolving context as agents coordinate and ex...

Sheng-Xuan Qiu, Hao-Chen Huang, Shu-Zhang Zhong et al. · 0 citations
Book Open access Sep 2026

Agents Break the Scheduler: Why ML-Based Resource Managers Fail for Agentic Workloads

ML-based resource managers—learned autoscalers, utilization-driven schedulers, RL-based controllers—were designed for workloads with stationary resource profiles. AI agents executing tool calls via the Model Context Protocol (MCP) violate every assumption these systems rely on: resource demand is non-stationary within...

Sheetal Sriram · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.