Scaling SDN Control Planes with Multi-Agent Reinforcement Learning

Abstract

Modern Wide-Area Software-Defined Networks (WAN-SDNs) face critical challenges regarding scalability, control-plane propagation latency, and single-points-of-failure due to their reliance on centralized control architectures. While logically distributed SDN controllers mitigate structural vulnerabilities, existing static or heuristic-based controller loadbalancing and inter-domain routing protocols cannot adapt to highly dynamic, asymmetric traffic demands. This paper presents an Intelligent, Distributed Control Plane framework driven by Multi-Agent Reinforcement Learning (MARL). In the proposed architecture, each distributed SDN controller operates as an autonomous deep Q-learning agent that continuously monitors local domain topologies and dynamically optimizes flow-rule allocations with neighboring controllers. To mitigate inter-controller communication overhead and control-channel flooding, we implement an Independent Double Deep Q-Network (IDDQN) variant augmented with a spatially discounted global reward function, explicitly preventing localized routing loops and policy divergence. We emulate our architecture on the Mininet platform utilizing the standard Internet2 topology. Empirical results demonstrate that our MARL-driven framework reduces average flow-setup latency by up to 28%, enhances network throughput by 19%, and accelerates fault-recovery times by 41% compared to legacy centralized control models, static distributed routing benchmarks, and standard MARL baselines, presenting a resilient, self-optimizing control topology for next-generation distributed networks.