DRLF: Deep Reinforcement Learning-Based Fragmentation-Aware Routing and Spectrum Allocation in Elastic Optical Networks

Abstract

Fragmentation poses a significant challenge in elastic optical networks (EONs) and diminishes resource utilization. While various research efforts have attempted to address the fragmentation issue, they often rely on rule-based strategies. Although these strategies encode valuable knowledge, they may not fully capture the dynamic and multifaceted behaviors of EONs. This limitation impedes adaptive service provisioning to mitigate fragmentation. To enhance network performance, this paper proposes a deep reinforcement learning-based fragmentation-aware routing and spectrum allocation, named DRLF, which employs deep neural networks (DNNs) to learn fragmentation-aware routing and spectrum allocation (RSA) within the intricate EON state spaces. Through the utilization of the deep Q-network (DQN) algorithm, DNN parameters are updated to facilitate episode-based training of DRLF. The RSA training process is segmented into episodes, each comprising a fixed number of lightpath requests, with a primary focus on optimizing network resource utilization while minimizing fragmentation. Unlike previous approaches, DRLF implements a continuous reward policy tied to path fragmentation, where rewards are inversely correlated with fragmentation levels. Additionally, the agent receives a bonus reward for successfully achieving the blocking probability under the threshold value in each episode. DRLF incorporates fragmentation as heuristic information in the reward function, and the action space is designed to train the agent to find a suitable spectrum allocation so that the fragmentation increase can be minimized. Consequently, the DRLF agent is trained to prioritize paths and spectrum slots with lower fragmentation levels, thereby accommodating more lightpath requests in future scenarios. Numerical evaluations demonstrate that DRLF surpasses existing DRL-based approaches, such as DeepRMSA and HeuDRL, as well as heuristic rule-based allocation strategies, particularly in terms of blocking performance.