Safe-ARCS: Risk-Budget Shielding for Hierarchical Reinforcement Learning in Automated Cybersecurity Incident Response
Abstract
Automated incident response should contain malicious activity without imposing unacceptable disruption on legitimate services. We study this conflict in a reproducible six-step stochastic simulator with noisy alerts, five latent incident classes, immediate containment actions, and persistent strategic defenses. Safe-ARCS builds on two mechanisms of the ARCS framework: independently learned immediate and strategic decision streams with exponentially decayed alert urgency, and temporal-difference prioritized replay. Its new budget shield admits an action only when a verified upper envelope on induced operational cost fits the remaining episode budget. For the specified bounded-cost simulator, this filtering yields a simple zero-overspend invariant. Across five independent training seeds and 59,500 held-out evaluation episodes, Safe-ARCS achieves 86.4% attack resolution and zero budget exceedances in nominal tests, versus 88.2% resolution and 0.5% exceedances for its unshielded dual-stream counterpart. Under shifted alert distributions and weakened mitigation probabilities, the corresponding results are 77.3% versus 80.4% resolution, with zero versus 1.9% exceedances. Ablations expose component effects. A separate 16,200-episode cost-misspecification study shows that under 50% inflation of modeled intervention costs, the default shield exceeds the budget in 3.7% of shifted-test episodes, whereas a calibrated 1.5-fold reserve eliminates these violations at a 2.1-point resolution cost. The findings define an auditable, conditional safety–effectiveness trade-off rather than an estimate of operational-network performance.