Robustness-Aware Machine Learning for Network Intrusion Detection under Feature and Distribution Variations Using UNSW-NB15

Abstract

Network intrusion-detection models are commonly evaluated with a fixed training and testing configuration. Such an evaluation does not fully show how a classifier behaves when feature values or class distributions change. This paper evaluates the robustness of a Random Forest model using the UNSW-NB15 dataset. The experiments examine controlled feature perturbation, reduction of the encoded feature representation, variation in the training attack proportion, and repeated sampling. The baseline accuracy is 87.14%. Under 20% feature perturbation, accuracy falls to 79.08% and MCC falls from 0.755 to 0.611. The top 25% encoded feature representation retains performance close to the complete representation. Changing the training attack proportion also changes the precision-recall balance; at 90% attack proportion, recall reaches 99.95%, while precision and MCC decrease. Repeated sampling gives relatively small standard deviations across the reported metrics. The results show that a single accuracy value is not sufficient to characterize intrusion-detection reliability and that robustness should be examined under controlled changes in data characteristics.