Optimization of Transmit Power and Clear Channel Assessment Threshold for Dense Wi-Fi Networks: A Distributed Reinforcement Learning Approach
Wi-Fi, that is wireless networks based on the IEEE 802.11 standard, operates in a decentralized manner based on carrier sense multiple access (CSMA). Owing to the operational characteristics of a CSMA protocol, the effects of interference and channel sensing sensitivity on the overall network throughput and fairness become more significant as the Wi-Fi network gets denser, i.e., the number of access points (APs) increases. The transmit and receive coverage can be adjusted by controlling the transmit power and clear channel assessment (CCA) threshold, respectively; the network performance can then be improved in terms of the network sum throughput and fairness. However, the analytical optimization of the transmit power and CCA threshold is a complicated task because both parameters of multiple APs and stations (STAs) are mutually coupled. Alternatively, the mechanism of the proposed problem is modeled using a Markov decision process (MDP) and the optimal solution is obtained by using a reinforcement learning (RL) approach. Considering the complexity and convergence rate of an algorithm as well as the distributed Wi-Fi network architecture, we propose a distributed multi-agent Q-learning algorithm. The effectiveness of the proposed algorithm is examined through intensive simulations with several benchmarks. Based on the simulation results, it can be deduced that the quality of services (QoS) of dense Wi-Fi networks can be effectively optimized by controlling the transmission power and CCA threshold.