Decay Driven Multi Objective Optimization for HPC nodes

Abstract

Energy efficiency has become a primary design constraint in modern high-performance computing systems, where energy costs increasingly dominate total cost of ownership. At the same time, applications are subject to strict performance requirements, making energy optimization inherently a multi-objective problem involving conflicting objectives of minimizing energy consumption while preserving application performance [1, 2]. Hardware interfaces such as Intel RAPL enable runtime power capping, allowing dynamic control over processor power budgets, but selecting optimal operating points remains challenging due to application-dependent and non-linear performance behavior [5, 7]. This work presents Decay-Driven Multi-Objective Reinforcement Learning (DDMORL), an offline preference-driven reinforcement learning framework for adaptive power control in HPC systems. The proposed approach extends preference-driven multi-objective reinforcement learning [3] to a fully offline setting [9], eliminating the need for unsafe online exploration and enabling deployment in production environments. Unlike prior approaches that rely on fixed scalarization or multiple trained policies, DDMORL learns a single preference-conditioned controller that spans the entire energy performance trade-off space. A key contribution of this work is a decay-driven preference formulation that replaces abstract preference weights with a user-defined performance degradation parameter. Users specify a maximum tolerable decay, which is then analytically mapped to a preference vector aligned with the corresponding operating point on the Pareto front. This mapping leverages a domain-specific monotonic relationship between application progress and power cap, removing the need for interpolation models typically required in preference-driven MORL.