Skip to content
Conference

Architectural Patterns for Scalable Multi-Agent LLM Systems: A Software Engineering Perspective

Aug 2026 · 2026 International Conference on Future and Intelligent Networking (FINE) · pp. 223-228 · 0 citations · 17 references

Abstract

The proliferation of large language model (LLM)-based autonomous agents has created a new class of distributed system: the multi-agent LLM network. While significant research focuses on the intelligence of individual agents, comparatively little work addresses the software architectural concerns that govern how fleets of such agents are composed, orchestrated, and scaled. This paper identifies and formalizes four recurring architectural patterns (Hierarchical Orchestration, Event-Driven Agent Mesh, Capability-Indexed Agent Registry, and Stateless Execution with Persistent Memory Offload), drawing on well-established technologies including Apache Kafka, MongoDB, AWS, Spring Boot, and Docker. All experiments were conducted exclusively on synthetically generated data using a locally hosted reference implementation on a personal development machine. No enterprise, production, or personally identifiable data was involved in this work. We analyze each pattern with respect to throughput, fault tolerance, observability, and latency trade-offs using a fully controlled synthetic workload, ensuring complete reproducibility. Our evaluation demonstrates that no single pattern dominates across all dimensions, and that composition of patterns yields the strongest results. This work provides a foundation for treating multi-agent LLM system design as a first-class software engineering discipline. Note: The four patterns operate at complementary abstraction levels coordination (Patterns 1 and 2), discovery and routing (Pattern 3), and state management (Pattern 4) and are designed for deliberate composition rather than as parallel alternatives.

View source

Similar papers

Preprint Oct 2026

Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows

This paper presents a comprehensive empirical study of agentic LLM systems across five software engineering tasks: code generation, technical debt identification, code vulnerability detection, log parsing, and log analysis, and translates these findings into design guidelines for sustainable, task-aware LLM-based devel...

Merve Astekin, Y. N. Tun, Arda Goknil et al. · 0 citations
Review 2026

The Systems Architecture of LLM Multi-Agent Systems: Routing, Memory, and Resource Optimisation

This survey presents a systematic taxonomy and technical review of dynamic orchestration strategies designed to address communication overhead, KV cache management challenges, and increased token consumption within large Language Model-based Multi-Agent Systems.

Heet Nagoriya, H. Raithatha · 0 citations
Open access Sep 2026

From DevOps to XOps: an agent-driven reference architecture for autonomous enterprise operations

XOps is proposed, a five-layer reference architecture integrating PlatformOps, DataOps, MLOps and AIOps beneath an Agentic Orchestration layer with Policy-as-Code governance, together with a continuous-time Markov chain model quantifying the availability effect of agent-driven remediation.

Mete Köse, E. Küçüksille · 0 citations
#small language model Open access Oct 2026

Agentic AI-Driven Data Engineering: A Framework for Self-Directed Enterprise Data Pipelines

Enterprise data pipelines have grown in scale and interdependency faster than the operational practices used to maintain them. Pipeline failures are still predominantly diagnosed and remediated by human engineers reading logs, tracing lineage, and applying tacit knowledge, which produces long mean-time-to-recovery and...

A. Kolla, Ameya Kokate · 0 citations

PANDA: A Decentralized Architecture with Flexible Orchestration for Scalable, Fault-Tolerant Multi-Agent Systems

Existing architectures for LLM-based multi-agent systems (MAS) cannot reliably and efficiently solve multi-step tasks at scale: they struggle to support large numbers of agents and concurrent tasks, tolerate failures, govern agent interactions, and accommodate the diverse planning and execution patterns different tasks...

Matthew Laws, Cristina Nita-Rotaru · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.