Hierarchical Reinforcement Learning with Hungarian Assignment for Reliable Urban Smart Metering Under Cognitive Spectrum Access
TL;DR
This work proposes a two-timescale hierarchy that couples these established tools in a new way: a Proximal Policy Optimization (PPO) agent decides, once per epoch, which opportunistic channels to expose, and an exact Hungarian solver performs the per-frame DAP-to-channel assignment.
Abstract
Advanced metering infrastructure (AMI) is the sensing backbone of the smart grid, and its reliability underpins urban energy services such as state estimation, demand response, and distributed-energy integration. When AMI uses cellular spectrum leased through a cognitive mobile virtual network operator (C-MVNO), allocating channels to data aggregation points (DAPs) each frame is difficult because three uncertainties interact: imperfect spectrum sensing, time-varying and cross-channel-correlated primary-user activity, and stochastic urban propagation. Classical Hungarian assignment is optimal per frame but blind to primary-user dynamics, while cognitive-radio heuristics ignore queue state and cross-channel structure. We propose a two-timescale hierarchy that couples these established tools in a new way: a Proximal Policy Optimization (PPO) agent decides, once per epoch, which opportunistic channels to expose, and an exact Hungarian solver performs the per-frame DAP-to-channel assignment. To our knowledge this is the first coupling of a learned cognitive layer with exact Hungarian assignment for cognitive-radio resource allocation. On a 3GPP TR 38.901-compliant simulator, PPO significantly outperforms a Bayesian-belief baseline and the Hungarian-only configuration in delivery ratio, latency, and a strict per-meter satisfaction metric, and is robust across independent seeds and sensitivity sweeps. An architectural ablation shows the DAP tier is a precondition for viability, not merely an optimization.