PPO-based QoS-aware resource management for multi-class URLLC and eMBB slices in NR/NR-U HetNets

Abstract

Exploiting new radio (NR) and NR-unlicensed (NR-U) technologies within a multi-RAT heterogeneous network can leverage traffic congestion and enhance overall capacity. thus, an optimization problem is introduced and formulated to maximize users’ quality of service (QoS) satisfaction by maximizing data rates while satisfying critical latency requirements. To accommodate heterogeneous URLLC services, multiple traffic classes with different delay requirements are considered, and a scalable numerology mini-slot technique is employed to satisfy the QoS requirements of each traffic class. An iterative framework is presented that combines regret matching learning for user association considering coexisting Wi-Fi users and a proximal policy optimization (PPO) algorithm which determines the optimal resource distribution based on a predefined numerology value for each slice. A logarithmic utility function is presented to enforce proportional fairness while satisfying each user’s service requirements. Simulation results demonstrate that the proposed framework achieves an overall QoS satisfaction rate of 74% significantly outperforming benchmark architectures including NR-only, NR-Wi-Fi, NR-NR-U, and LWA. Furthermore, comparative evaluations against other reinforcement learning algorithms—such as Q-network, deep Q-network (DQN), double DQN, dueling DQN, and actor-critic confirm stability and performance of the proposed PPO-based approach.