Skip to content
Log in
Sponsored Design Multi-Agent AI Systems Using MCP and A2A: Engineer your own Pyt Design Multi-Agent AI Systems Using MCP and A2A: Engineer your own Pyt

Author

Hai-Ying Shen

h-index 4 View on Semantic Scholar

We have 4 of 26 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Book Open access Sep 2026

Heterogeneous SLO Guaranteed Multi-Resource-Aware Batching in LLM Serving

In this paper, we study a mixed-prompt scenario—where both short and long prompts coexist—in an LLM inference serving system that supports diverse applications with heterogeneous iteration-time SLOs. To improve throughput for long prompts, prior work divides them into chunks and batches requests or chunks to meet the t...

Hai-Ying Shen, Tanmoy Sen, Yuxiong He · 0 citations
Conference Jul 2026

Managing KV Cache for Coordinated Waiting and Execution Time in LLM Serving

Large Language Model (LLM) serving systems face a KV-cache (KVC) bottleneck. In this paper, our experimental study shows that block-based allocation increases Time-Between-Tokens (TBT) due to preemptions, while prediction-based allocation increases Time-to-First-Token (TTFT) and TBT due to allocated but unused KVC and...

Haiying Shen, Tanmoy Sen, Masahiro Tanaka · 0 citations
Conference Jul 2026

WaSMa: Workflow-Aware Scheduling for Multi-Agent LLM Systems

Public LLM services serve diverse multi-agent applications with varying workflow dependencies and performance requirements. Requests generated by these applications often exhibit commonality and interdependence, yet current systems largely ignore such application-level structure. As a result, at the LLM engine cluster...

Uttam Rao, Ali Zafar Sadiq, Hai-Ying Shen et al. · 0 citations
Conference Jul 2026

FlowGuard: Slack-Aware Overload Control for Multi-Agent LLM Serving

Multi-agent applications increasingly rely on shared large language model backends in the public cloud, where bursty workloads cause requests from different agents to contend for the same LLM instances, leading to long queues, memory imbalance, and severe tail-latency inflation. Existing approaches typically prioritize...

Ali Zafar Sadiq, Hai-Ying Shen · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.