A Comprehensive Survey on Machine Learning and Deep Learning Approaches for Network Intrusion Detection Systems: Techniques, Datasets, and Open Challenges

Abstract

The rapid expansion of networked infrastructure, cloud computing, and IoT environments has significantly increased the attack surface, while traditional signature-based Network Intrusion Detection Systems (NIDS) remain limited in detecting zero-day attacks, polymorphic threats, and malicious activities within encrypted traffic. This survey reviews and synthesizes a decade of research from 2015 to 2025 on Machine Learning (ML) and Deep Learning (DL) approaches for NIDS, providing a unified taxonomy that organizes existing methods according to both their learning paradigms, including supervised, unsupervised, semi-supervised, reinforcement, and self-supervised learning, and their architectural designs, such as classical ML, convolutional, recurrent, autoencoder, generative, attention-based, transformer, graph, and federated models. The review shows that tree-based ensemble methods, particularly Random Forest and XGBoost, continue to provide strong and competitive baselines for tabular network-flow data, highlighting the need for more complex DL models to demonstrate clear practical advantages. Hybrid spatio-temporal architectures, such as CNN-LSTM models, and transformer-based approaches have achieved strong performance on recent datasets; however, results on saturated benchmarks such as NSL-KDD provide limited evidence of meaningful progress. More importantly, cross-dataset evaluations reveal substantial generalization gaps, with F1-score differences of approximately 20–30 percentage points, indicating persistent problems related to dataset bias and label quality. Overall, NIDS research is gradually moving beyond the pursuit of higher accuracy toward practical challenges such as real-time edge inference, adversarial robustness, explainability, privacy-preserving and federated learning, and intrusion detection over encrypted traffic. Based on these findings, this survey highlights standardized feature representations, rigorous temporal and cross-dataset evaluation, and self-supervised pre-training as promising directions for developing more robust and generalizable NIDS models.