VERA: Reinforcement Learning for Dynamic Memory Scaling of HPC Workloads in Kubernetes
TL;DR
A reinforcement learning (RL) recommender VERA is presented that formulates vertical memory scaling as a Markov Decision Process and trains an agent on 3353 real Prometheus traces and demonstrates that an observation-driven RL recommender could outperform retrospective heuristics for dynamic memory scaling.
Abstract
Memory over-provisioning results in resource underutilization when HPC workloads run on Kubernetes. The default Vertical Pod Autoscaler (VPA) cannot anticipate phase-driven memory spikes for first-run HPC jobs. In this work, we present a reinforcement learning (RL) recommender VERA that formulates vertical memory scaling as a Markov Decision Process and trains an agent on 3353 real Prometheus traces. Evaluated on a live Google Kubernetes Engine cluster using LAMMPS, graph analytics, in-memory analytics, and MLPerf 3D-UNet, the RL agent reclaims 31.6% of the available memory headroom and incurs at most one OOM event while VPA reclaims -7.9% over the same runs, raising memory provisioning, and its recommendation would have been insufficient to avoid OOM in 30 runs. The results demonstrate that an observation-driven RL recommender could outperform retrospective heuristics for dynamic memory scaling.