A Queue-Aware Scheduling Framework with Reinforcement Learning in Massive MIMO

Abstract

To support multiple users with quality-of-service (QoS) requirements over shared wireless resources, power control and scheduling are essential. While reinforcement learning (RL) has been applied to learn transmit powers directly, this approach may lack robustness in time-varying channels where optimal power depends on instantaneous channel and traffic states. This paper proposes a decoupled framework for queue-aware resource allocation. Instead of learning powers, RL aims to learn a mapping from queue lengths to the weights of a weighted sum-rate maximization problem to capture long-term dynamics, while transmit powers are computed per slot via optimization. That is, by decoupling, the RL agent can focus on long-term queue stability without being obscured by channel fluctuations. Furthermore, a hybrid law is introduced to revert to the robust MaxWeight baseline during unstable early training stages. Simulation results demonstrate that the proposed framework achieves superior queue stability and significantly lower maximum queue lengths compared to the conventional MaxWeight approach.