Safe Reinforcement Learning With Deep Q-Networks for Energy-Efficient Sleep Scheduling in Wireless Sensor Networks

Abstract

Wireless sensor networks are widely used in Internet of Things applications where energy efficiency and reliable packet delivery must be jointly considered. This paper formulates energy-aware sleep scheduling in multi-hop wireless sensor networks as a constrained Markov decision process, where long-term node activity is reduced subject to an explicit end-to-end packet delivery ratio requirement. To address this problem, a safety-aware deep Q-learning framework named SafeDQN is proposed. The method combines a deep Q-network with a Lagrangian dual update mechanism that penalizes reliability violations during learning while allowing the controller to reduce unnecessary node activation. Under nominal 50-node conditions, SafeDQN satisfies the target packet delivery ratio, whereas Random Scheduling, LEACH-like, PEGASIS-like, and UnconstrainedDQN remain below the target. PenaltyDQN also satisfies the reliability requirement and is further compared with SafeDQN through paired evaluations and penalty-coefficient sensitivity analysis. A policy trained on 50-node networks under the strict reliability target is evaluated without retraining on deployments with 30, 50, 80, 500, and 1500 nodes. A separately trained relaxed-target policy is evaluated on deployments with 30, 50, 80, and 1500 nodes, with constraint satisfaction varying across the tested configurations. Additional experiments under dynamic traffic and probabilistic link reception characterize performance under environmental variation. These results indicate that constrained deep reinforcement learning is a promising simulation-based approach for reliability-aware energy management in wireless sensor networks under the stated modeling assumptions.