Autoencoder Residual Augmented XGBoost with HDBSCAN Clustering for Zero-Shot Network Intrusion Detection on CIC-IDS-2017

Abstract

A Network Intrusion Detection System (NIDS) must be able to detect zero-shot attack classes not present in their training set. Traditional supervised models cannot do this task, and purely unsupervised methods generate too many false positives. This paper describes a hybrid approach consisting of a deep autoencoder trained on benign samples, Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) clustering of latent space using Uniform Manifold Approximation and Projection (UMAP) for pre-reduction, together with an augmented XGBoost classifier. The model is tested on the dataset CIC-IDS 2017, in which Bot and Web Attack classes are entirely removed from the training set to evaluate zeroshot capabilities. An isolation forest represents the fourth detector. Decisions made by the model are explained through SHapley Additive exPlanations (SHAP) explanations and then transformed into human-friendly language through a natural language generation module.