Hybrid Learning Framework for Throughput Enhancement in IoT Networks With Semi-Grant-Free NOMA

Abstract

Semi-grant-free non-orthogonal multiple access (SGF-NOMA) schemes group one grant-based (GB) user with multiple grant-free (GF) users into one time/frequency resource block (RB) to enhance spectral efficiency. Due to the sporadic traffic of GF users and the stringent quality of service (QoS) requirement of the GB user, the access collision problem becomes severe in SGF-NOMA. To solve this problem, this paper firstly designs an RB-based power pool (PP), which directs GF users to adjust their transmit power without disrupting the ongoing transmission of the internal GB user. After that, this work proposes an efficient multi-agent deep reinforcement learning (MA-DRL) framework to jointly optimize the PP and access strategy for maximizing the network throughput. In particular, this work exploits the fast-response feature of the traditional competitive MA-DRL and the increased-performance feature of the traditional cooperative MA-DRL to redesign a mixed reward system, which contributes to a hybrid MA-DRL mode for enhancing the learning efficiency of agents, i.e., GF users. We investigate the performance of the proposed algorithm at the network level and the NOMA-cluster level. We show that the proposed hybrid MA-DRL at the cluster level converges faster to an optimal solution than that at the network level but at an extra cost of user clustering. The numerical results show that the proposed scheme increases the successful decoded users by 42.38% when compared to the traditional schemes without learning capability. The proposed hybrid MA-DRL mode performs better than the pure competitive and cooperative MA-DRL modes, especially under a heavy-load network. It is able to achieve a 69% success rate of access in a time-varying environment with high packet arrival rates.