2026· International Journal of Knowledge Based Computer Systems· 0 citations
TL;DR
This survey presents a systematic taxonomy and technical review of dynamic orchestration strategies designed to address communication overhead, KV cache management challenges, and increased token consumption within large Language Model-based Multi-Agent Systems.
Abstract
Large Language Model (LLM)-based Multi-Agent Systems (MAS) have attracted significant research attention due to their ability to coordinate specialised agents for complex reasoning tasks. Their deployment within static, sequential multi-agent pipelines introduces communication overhead, KV cache management challenges, and increased token consumption. Recent empirical studies report that iterative refinement stages can account for up to 59.4% of total token consumption in software engineering workflows, with an average 2:1 input-to-output token ratio highlighting the impact of communication overhead in conventional architectures. This survey presents a systematic taxonomy and technical review of dynamic orchestration strategies designed to address these constraints. We examine routing and orchestration approaches including Directed Acyclic Graph (DAG)-based frameworks, semantic routing methods, graph-based optimisation techniques, and search-driven workflow planners, analysing their decision logic, computational overhead, and infrastructure trade-offs. We further review distributed state management techniques—including structurally lossless context trimming, tiered agentic memory systems, and test-time error pruning—that aim to mitigate state accumulation and reduce error propagation. A dedicated analysis of cost-aware routing via linear programming highlights the potential benefits of difficulty-aware model dispatch. Finally, we discuss hardware-aware routing and automated failure attribution as promising directions for future research. The surveyed literature highlights the growing importance of orchestration strategies, routing mechanisms, resource management techniques, and system-level evaluation methodologies for scalable deployment.
G-STAR is a general graph-based scheduling framework that formalizes complex MAS pipelines as attributed Directed Acyclic Graphs (DAGs) and develops an industry-grade orchestration stack with asynchronous execution, resilient serving, and audit-friendly artifacts, offering a practical solution for optimizing web-scale deployments of complex MAS pipelines.
Jiabao Song, Yunsheng Xia, Beibei Kong et al.· Proceedings of the 32nd ACM...· 0 citations
Public LLM services serve diverse multi-agent applications with varying workflow dependencies and performance requirements. Requests generated by these applications often exhibit commonality and interdependence, yet current systems largely ignore such application-level structure. As a result, at the LLM engine cluster level, assigning requests to engines with the shortest queue can cause inefficient KV-cache transfers across GPUs. Using three representative multi-agent applications, we show that current scheduling methods miss opportunities to (a) improve performance through KV-cache reuse and reduced data transfer, and (b) increase goodput via batch management informed by workflow dependencies. Motivated by these observations, we propose a Workflow-Aware Scheduling system for Multi-Agent LLM systems (WaSMa) that incorporates cluster-and engine-level scheduling to optimize LLM request execution across GPU resources. Experimental results show that WaSMa reduces the P95 tail latency by up to 48% and improves goodput by up to 107% compared to existing methods.
Uttam Rao, Ali Zafar Sadiq, Haiying Shen et al.· International Conference on...· 0 citations
In Federated Learning (FL), the communication topology is a runtime variable rather than a fixed design choice, since links and edge devices drop in and out during training. Each round, the server must commit three coupled decisions, namely the communication topology, per-client resource allocation, and the aggregation rule for combining local updates. Recent agentic systems have begun bringing large language models (LLM) into FL, but the existing line of work either operates at setup time or handles a single runtime dimension such as client selection. We propose FL-MAESTRO, a multi-agent orchestrator that makes the joint runtime FL decision directly through three specialist LLM agents, one per decision dimension. A coordinator combines their analyses into a single decision, and a non-LLM feasibility check confirms it before the round executes. Because the orchestrator consumes the server's predicted-failure list, it withholds clients whose updates would never be aggregated, which removes the dominant source of wasted round energy in classical FL on volatile edge networks. Because client state is read as natural-text profiles, the same orchestrator extends to heterogeneous device classes without per-class energy models. On a non-IID CIFAR-10 benchmark, FL-MAESTRO matches the accuracy of the strongest energy-aware baseline while cutting wasted round energy from over a third to near zero. Code is available at https://github.com/denoslab/FL-MAESTRO.
Jiajun Wu, Zirui Wang, Jiayu Zhou et al.· 0 citations
LLM-based multi-agent systems have the potential to enable collective intelligence and scale toward solving highly complex tasks through coordinated ensembles of specialized agents. However, despite their theoretical potential, the architectural design space remains largely non-systematized and lacks broadly established design principles. Furthermore, the scalability characteristics of such systems are only partially understood so far. This paper makes two contributions. We first distill four design principles for scalable MAS architectures from a structured analysis of prior work: simplicity, elastic feedback, sequential workflows with optional loops, and summary-based communication. We operationalize these principles in a reference architecture whose topology is formalized as a constrained directed workflow graph, and we evaluate four configurations of increasing complexity on a standardized benchmark of terminal-based system engineering tasks using two LLMs of differing capability. Our findings show that scaling yields measurable accuracy improvements with approximately linear cost growth, but only when the underlying LLM exceeds a minimum capability threshold. Performance peaks at intermediate complexity, then degrades due to timeouts and evaluation limitations. In addition, persistent consistency issues emerge as a central challenge across all scaling levels. These results provide concrete design guidance for practitioners and highlight consistency and evaluation standardization as key targets for future research.
Linus Sander, F. Pan, Vahid Zolfaghari et al.· 0 citations
LAMaS is a latency-aware orchestration framework for learning-based multi-agent systems that achieves the best latency among evaluated learning-based MAS baselines, reducing end-to-end latency by over 50% while maintaining competitive or better accuracy.
The rapid expansion of capabilities in Large Language Model (LLM) agents has exposed a critical architectural bottleneck: when agents are given access to a flat, monolithic registry of tools, the model must evaluate hundreds or thousands of options simultaneously. This leads to decision-space explosion, context window saturation, and degraded routing accuracy. To address these limitations, this paper presents a hierarchical, skill-based architecture for agentic orchestration. Capabilities are organized as a rooted tree where internal nodes make routing decisions and leaf nodes execute deterministic tasks. The runtime enforces a single-step execution loop governed by a Last-In-First-Out (LIFO) stack, giving the agent a form of memory akin to a Pushdown Automaton, therefore enabling it to track nested execution contexts and resume deterministically from any depth. Capability discovery follows a manifest-driven, lazy-loading protocol: only the immediate children of the active node are loaded, so memory and prompt costs scale with the explored path rather than the global registry. By replacing global memory with localized stack frames, the architecture prevents outputs from one execution branch from leaking into another, establishing the isolation guarantees required for deployment in regulated enterprise environments. We also discuss UPI Help, an AI-powered digital payments support product, as a motivating production deployment context. We provide a mathematical formalization of the orchestration state, detailed algorithmic analysis of the execution loop, and controlled benchmarks comparing flat and hierarchical routing under increasing tool catalogs, multi-step workflow pressure, and visible schema-token exposure per LLM call.
Prashant Devadiga, Abhishek, Adithya Mishra et al.· 0 citations