Skip to content
Log in
Sponsored Artificial Intelligence: A Modern Approach, Global Edition Artificial Intelligence: A Modern Approach, Global Edition
Book Open access

Scaling LLM Agent Tool Access at Cloud Scale

Ming-Xing Li Enge Song Yueshang Zuo Xiaodong Liu Rong Wen Jian-Bao He Tie Jing Zhou Shao Qiang Fu Xiaobo Xue Xiong Xiao Luya Zhong Shao-Kai Zhang Jian-Gu Zhao Jian-Yuan Lu Shize Zhang Xiaoqing Sun Chang-Gang Zheng Zi-Hao Fan Hao-Nan Li Tian Pan Xiao-Min Wu Yang Song Xing Li Biao Lyu Meng Li Hai-Peng Dai Guihai Chen Shunmin Zhu
Aug 2026 · Asia-Pacific Workshop on Networking · pp. 304-307 · 0 citations · 6 references
Computer Science

TL;DR

The gateway breaks the direct-connect data plane and consolidates legacy API integration, protocol bridging, access control, and session-aware routing, while scaling out elastically at low per-call overhead.

Abstract

LLM agents increasingly rely on tool calling, and the Model Context Protocol (MCP) standardizes it between agents and tool providers, reducing integration cost and driving rapid growth in tool scale. Yet a standardized interface does not make tool access work at production scale: legacy services are not MCP-callable, fast protocol evolution creates compatibility cost, large tool sets exhaust the context window, and stateful sessions complicate load balancing. We solve these with a shared control point, a centralized MCP Gateway System that makes MCP operational at cloud scale. The gateway breaks the direct-connect data plane and consolidates legacy API integration, protocol bridging, access control, and session-aware routing, while scaling out elastically at low per-call overhead. It scales agent tool access to thousands of cloud operations.

DOI Semantic Scholar
Read PDF

Similar papers

Preprint Aug 2026

Hybrid Semantic Tool Discovery for Enterprise MCP Gateway: Architecture and Implementation

This work presents SCOUT (Selective Context Optimization for Universal Tooling for Universal Tooling), which reframes tool exposure as a context-selection problem, injecting only tools relevant to the current step, and reduces MCP tool-token consumption by 99%, cutting per-query inference cost at enterprise scale.

Olympia Saha, Amy Wang, Srinivasan Manoharan · 0 citations
Open access Sep 2026

FinDS-Agent: A Cloud–Edge Collaborative Data Science Agent for Financial Analytics

Large language model agents can automate data science workflows, but cloud-centric deployment exposes sensitive context and edge-only deployment limits analytical capability. We present FinDS-Agent, a cloud–edge framework that keeps raw records and program execution at the trusted edge while providing a policy-screened...

Xiao-Zheng Du, Rui-Jun Deng, Cheng Wang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems

Large Language Models (LLMs) with function-calling capabilities are becoming critical for modern agentic AI systems. Nevertheless, current deployments typically route inferences to powerful cloud-based models, incurring significant energy use and carbon emissions. We address this sustainability challenge with a carbon-...

Aikaterini Maria Panteleaki, Varatheepan Paramanayakam, S. Tragoudas et al. · 0 citations
Aug 2026

OMBench: Taming Data Management Requirements in Cloud-Native Applications

Function-as-a-Service, actor runtimes, and microservice frameworks have emerged as popular platforms for scalable cloud applications. However, understanding the data management trade-offs of these solutions, not to mention their programming abstractions and architectural design, can be cumbersome for practitioners and...

R. Laigner, Xikun Jiang, Boris Düdder et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.