Towards reliable intrusion detection in smart grids: A multiclass, process-aware perspective
TL;DR
A systematic evaluation of the Sherlock dataset proves that, when high-fidelity event labels are available, supervised multi-class learning provides a robust, accurate, and engineering-valid foundation for intrusion detection in the modern, renewable-integrated grids.
Abstract
The development of effective intrusion detection systems for renewable-integrated smart grids necessitates models that are capable of distinguishing between legitimate operational variations and genuine cyberattacks. In this study, we present a systematic evaluation of the Sherlock dataset, a recent benchmark for process-aware security research. The investigation encompasses three distinct learning paradigms: unsupervised anomaly detection, binary supervised classification, and a proposed multi-class supervised formulation. The results demonstrate that anomaly detection methods fail to separate attacks from benign operational anomalies due to structural characteristics of the dataset, including benign anomalous behavior and substantial drift between training and testing distributions. While binary supervised classification appears effective under a single split, its performance collapses once cross-validation is applied, with accuracy fluctuating significantly between 35% and 96% (σ ≈ 0.19), which indicates that its apparent success relies on favorable sampling rather than meaningful generalization. Conversely, the multi-class formulation, combined with duplicate removal, variance filtering, and top-80 feature selection, provides a stable and context-aware solution. A tuned eXtreme Gradient Boosting classifier achieves a mean accuracy of 99.81% ± 0.04% across 5 independent seeds successfully identifying all fifteen classes, including rare maintenance events and four distinct attack types. Feature importance analysis confirms that the model primarily relies on physically meaningful variables such as voltage, reactive power, and line currents, thereby demonstrating process-aware decision-making. The results establish that, when high-fidelity event labels are available, supervised multi-class learning provides a robust, accurate, and engineering-valid foundation for intrusion detection in the modern, renewable-integrated grids.