LLM agents increasingly rely on tool calling to act on external systems, and the Model Context Protocol (MCP) has quickly become its de facto interface. Operating MCP at cloud scale, however, becomes difficult. On the tool provider side, legacy services are not directly callable through MCP; the rapid protocol development also creates ongoing compatibility cost. On the agent side, the number of accessible tool is limited by the LLM context window and inference overhead; mounting a large tool set increases token usage and inference latency and can reduce task success rate. Moreover, for stateful MCP backends with multiple replicas, preserving session affinity increases client-side complexity. We present a cloud-scale gateway system for MCP service. It breaks the direct-connect model on the data plane and offloads legacy service integration, consolidating incompatible MCP variants, access control, tool recommendation, and session-aware routing to the gateway. Hybrid retrieval sustains 98% Top-15 recall; it scales agent tool access to 3,000+ with high tool selection accuracy, and reduces tool selection time by $8.9\times$ and token usage by $23.8\times$, with low per-call overhead, stable under scale-out. Finally, we share the lessons learned from deploying the gateway system in production.
Mingxing Li, Enge Song, Yueshang Zuo et al.· 0 citations
Since the inception of modern communication networks, the quest for operations automation has never ceased. Yet the evolution of network automation is difficult to characterize with a single maturity ladder. Throughout this history, network control systems have expanded their capabilities for observation, decision support, routine execution, and operator interaction, but these capabilities have not advanced uniformly. Such uneven progress makes the degree of automation an unreliable proxy for trustworthy network-side actuation. The unresolved question is not simply how much automation a system provides, but under what conditions it can be entrusted to change the network state. This paper examines that question through Network Control Intelligence (NCI), a five-axis framework spanning Decision Logic, Adaptability, Knowledge, Control Delegation, and Interface. We use NCI to organize the evolution of network-control systems into three eras: rule-based and scripted automation, programmable and data-driven control, and Large Language Model (LLM)-enabled network operations. Viewed through this framework, the three eras reveal a persistent asymmetry. None of these gains, however, automatically determines when network control should be trusted to change the network state. We frame trustworthy autonomy as a governed alignment between what a system can infer, what it can verify, and what it is authorized to execute. On that basis, the paper develops a reference architecture that separates proposal generation from governed execution, identifies recurring integration patterns for LLM-enabled operations, and derives a research agenda for higher network autonomy under explicit assurance, safety, and governance constraints.
Tianzhu Zhang, Changgang Zheng, Shanshan Wang et al.· 0 citations
An architecture that decouples complex policy enforcement from high-speed packet forwarding to support VPC semantics on back-end NICs and enable front-end/back-end integration is proposed, suggesting that commodity hardware can support both high-throughput AI training and flexible VPC features.
Yinhe Wang, Xing Li, Enge Song et al.· Asia-Pacific Workshop on Net...· 0 citations
ParaFlowO is proposed, an architecture that Parallelizes processing elephant Flows across multiple CPU cores while preserving in-Order delivery and integrates a lightweight reordering mechanism to preserve packet order and controls parallelism to mitigate contention on shared state.
Changgang Zheng, Bowen Yang, Jin Ke et al.· Asia-Pacific Workshop on Net...· 1 citation
