Deep Recurrent Q-Learning Based Beam Steering Strategy for Throughput Maximization in WPCNs

Abstract

In wireless powered communication networks, medium access control protocols for devices using the harvest-then-transmit strategy must be distributed, low-overhead, and capable of handling irregular and infrequent data transmissions to ensure efficient energy utilisation. However, most existing protocols fail to meet one or more of those requirements, leading to wasted scarce harvested energy. We address this by identifying beam steering as a potential mechanism to regulate the charging rate of energy harvesting devices and thus control their access to the shared wireless medium. After formulating a joint problem of energy beam steering and slotted ALOHA-based random access, we leverage a deep learning framework based on an action-specific deep recurrent Q-Network (ADRQN) to learn a beam-steering policy only from the macro-level ternary slot outcomes, namely, idle, success and collision. Additionally, we design an oracle policy with global knowledge of the network to benchmark our proposed blind adaptive beam-steering approach. The numerical results demonstrate that our approach achieves up to 68\% increase in throughput compared to non-learning schemes, while also reaching 75-80\% of the oracle policy's performance, all without requiring channel estimation, charge-level reporting, or device-state tracking.