Deep Reinforcement Learning for Joint Communication and Control Design in Wireless Networked Control Systems

Abstract

This paper investigates a wireless networked control system where a base station (BS) controller coordinates multiple nonlinear dynamic subsystems over wireless fading channels. In the considered setup, sensors first upload state observations to the BS, which then processes the received information to generate control commands. These commands are subsequently downloaded by the actuators to update the plant states. The system faces two key challenges: the BS controller has no prior knowledge of the state transition functions of the subsystems, and the wireless fading channels introduce uncertainties in the transmission of both state information and control signals. These limitations render traditional model-based control-communication co-design methods inapplicable. To address these issues, we propose a data-driven approach that leverages deep learning and deep reinforcement learning (DRL) to maximize the cumulative reward across all subsystems. Specifically, we first employ long short-term memory (LSTM) neural networks to estimate missing observations and control inputs resulting from transmission losses. We then apply a deep deterministic policy gradient (DDPG) algorithm with prioritized experience replay to jointly optimize the control inputs and wireless resource allocation, thereby enhancing the overall control performance. Finally, simulations conducted on the OpenAI Gym platform demonstrate that the proposed approach significantly outperforms several benchmark schemes.