Interactive Resource Planning and Change Detection via Multi-agent Reinforcement Learning

Abstract

Rapid advancements in sensor technology generate vast amounts of data, posing challenges for statistical process control (SPC). Resource constraints, such as limited bandwidth and processing power, restrict real-time data observability, necessitating dynamic decisions on which data streams to monitor for prompt anomaly detection. Given the high costs of monitoring resources, it is essential to consider the operational expenses associated with online monitoring. To fill this research gap, we reformulate the problem of change detection and resource allocation as a leader-follower decision-making process, utilizing bilevel programming to reconcile the conflicting objectives of cost control and anomaly detection. Then, we present a generic framework of policy learning based on deep reinforcement learning (DRL) to optimize decision-makers' policies. To enhance scalability and mitigate the curse of dimensionality, we decompose the complex change detection task and employ multi-agent reinforcement learning for improved collaboration. Unlike existing methods, our approach prioritizes long-term returns over myopic outcomes, effectively addressing scalability issues in DRL. Moreover, we are the first to incorporate operational costs into change detection. Simulation studies and a real case study validate the efficacy and applicability of the proposed method.