Feasibility-Aware Reinforcement Learning for Reliable Hop-Constrained Routing in Wireless Sensor Networks
TL;DR
The experiments show that feasibility-aware learning can approach deterministic baseline reliability while retaining learned forwarding capability under hop constraints, and confirm that action masking is the dominant mechanism for maintaining feasible routing decisions, whereas trust mainly provides reliability-aware regularization.
Abstract
: Hop-constrained packet routing is a fundamental problem in wireless sensor networks (WSNs), where latency constraints, energy limitations, and practical feasibility requirements greatly restrict routing choices. Traditional methods based on shortest path and greedy routing have low complexity but cannot adapt to dynamic network changes well, while reinforcement learning for routing has the potential to adapt to network variations but has not been well explored in the hard hop-constrained setting. The current study attempts to fill the gap by modeling hop-constrained routing as the decision-making problem in a finite-horizon setting. An integrated simulation environment is proposed that unifies the concept of feasibility-aware action masking, energy-and trust-aware routing mechanisms, and simulation-related evaluation criteria. In this unified environment, four representative reinforcement learning methods, REINFORCE, Advantage Actor–Critic (A2C), Proximal Policy Optimization (PPO), and Deep Q-Network (DQN), are applied and validated against greedy forwarding, shortest-path routing, and Dijkstra routing under strict ( H = 5) and relaxed ( H = 15) hop limits using multi-seed testing. Under tight hop constraints, Dijkstra achieves a delivery success rate of 1.000, while greedy routing reaches 0.950 ± 0.014. Among the learning algorithms, PPO, A2C, and DQN reach approximately 0.945 ± 0.014 at their best checkpoints with near-baseline hop efficiency, indicated by an average hop count of about 4.34 ± 0.04. Under relaxed hop constraints, Dijkstra remains at 1.000, greedy forwarding reaches 0.984 ± 0.008, and PPO, A2C, and DQN achieve high best-checkpoint success rates of approximately 0.991– 0.992. REINFORCE improves under the relaxed setting but remains less stable than the stronger learned policies. The experiments show that feasibility-aware learning can approach deterministic baseline reliability while retaining learned forwarding capability under hop constraints. The ablation results further confirm that action masking is the dominant mechanism for maintaining feasible routing decisions, whereas trust mainly provides reliability-aware regularization. These observations emphasize the need to evaluate RL-based routing using deployment-level metrics, including success probability, hop-count distribution, invalid-action rate, route-risk rate, and return, rather than relying only on training reward.