Skip to content
Review

Artificial Intelligence for Malware Detection in Software-Defined Networks: A Comprehensive Systematic Literature Review

Jul 2026 · International journal of advanced innovative technology in engineering · pp. 18 · 0 citations · 24 references

TL;DR

The review outlines future research directions emphasizing lightweight and explainable AI models, graph neural networks, federated and continual learning, adaptive hybrid intelligence, and standardized real-world evaluation frameworks to support the development of accurate, scalable, robust, and deployable malware detection systems for next-generation Software-Defined Networks.

Abstract

Software-Defined Networking has emerged as a fundamental networking paradigm for cloud computing, Internet of Things, fifth-generation (5G) communication, and data-center infrastructures because of its centralized control, programmability, and flexible network management. However, the logical centralization of the control plane also introduces significant security vulnerabilities, making SDN increasingly susceptible to malware, botnets, ransomware, Distributed Denial-of-Service, and other sophisticated cyberattacks. Artificial Intelligence (AI)-based malware detection techniques have gained considerable attention due to their capability to identify complex and previously unseen attack patterns. This paper presents a comprehensive Systematic Literature Review of intelligent malware detection approaches for SDN by following the PRISMA 2020 framework and Kitchenham guidelines. A total of 30 primary studies published between 2020 and 2026 were systematically selected, assessed, and synthesized. The reviewed literature was classified into three major categories: Machine Learning, Deep Learning, and Hybrid AI approaches, followed by comprehensive comparative analyses of datasets, learning algorithms, feature engineering strategies, evaluation metrics, detection performance, and reported limitations. The quantitative synthesis indicates a clear research transition from conventional ML techniques toward deep learning and hybrid intelligence frameworks, with hybrid models consistently shows the highest detection performance, frequently exceeding 99% detection accuracy. The analysis further reveals that Random Forest, Support Vector Machine, Convolutional Neural Networks, Long Short-Term Memory networks, Deep Neural Networks, and CNN–LSTM hybrid architectures are among the most widely adopted algorithms, whereas NSL-KDD, InSDN, UNSW-NB15, CICIDS2017, and IoT-23 remain the dominant evaluation datasets. The review identifies several persistent challenges, including dependence on benchmark datasets, limited real-world SDN validation, class imbalance, high computational complexity, insufficient explainability, limited cross-dataset generalization, and the absence of standardized benchmarking protocols. The review outlines future research directions emphasizing lightweight and explainable AI models, graph neural networks, federated and continual learning, adaptive hybrid intelligence, and standardized real-world evaluation frameworks to support the development of accurate, scalable, robust, and deployable malware detection systems for next-generation Software-Defined Networks.

View source

Similar papers

#generative ai Review Open access Aug 2026

Artificial Intelligence for Real-Time Cyber Threat Classification and Emerging Threat Detection: A Structured Review of Methods, Datasets, Challenges, and Research Directions

The reviewed literature indicates that AI-based methodologies often demonstrate superior detection capabilities for intricate and previously unseen attack patterns compared to traditional methods; however, direct performance comparisons are complicated due to discrepancies in datasets, experimental designs, and evaluation protocols.

Jaswanth Garugu · 0 citations
Review Open access Aug 2026

Machine intelligence approaches for preventing adversarial malware attacks in intrusion detection systems: A systematic review

In academia, the epistemology of Adversarial Malware Attacks ( AMAs ) in the context of an Intrusion Detection System ( IDS ) has not been fully grasped. Using Machine Intelligence ( MI ), many attempts have been made to reproduce modeling techniques and methods within IDSs that can properly track, trace, and prevent AMAs from reappearing. However, there seems to be a lack of comprehensive and cohesive literature reviews in this area on which existing scholars can rely to pursue future research and broaden the field of Information Security and Networks ( ISN ) research. Motivated by the absence of a universal IDSs framework, termed a ’one-stop shop,’ this systematic review hopes to serve the research community by aligning thinking and firmly consolidating the historical knowledge and progress made in MI approaches, thereby providing a better understanding of AMAs, including their prevention and the implementation of viable and efficient IDSs. No other systematic review of this kind has yet been produced. Starting with nearly 1,000 papers and applying rigorous analysis, this study covers 132 research papers in multiple bibliographic databases since January 2020, highlighting studies on IDSs deployed for AMA detection and classification using MI learning techniques such as Machine Learning ( ML ) and Deep Learning ( DL ); methods involving feature extraction; studies discussing static, dynamic, memory, and hybrid feature analysis; and finally, studies providing experimental results and accuracy metrics from these IDSs. Throughout this comprehensive review, the emphasis is placed on highlighting the differences, strengths, and shortcomings from each MI-IDS approach, as well as providing discussions and suggestions for further exploration in future research. Following the PRISMA protocol, this systematic literature review (SLR) provides a rigorous, transparent, and reproducible foundation for knowledge by synthesizing all available evidence on IDSs over recent years.

Unknown authors · 0 citations
Review Open access 2026

A Systematic Review of Machine Learning Techniques in Intrusion Detection Systems

: Background: The evolution of modern networked systems in complexity, volume, and diversity has markedly increased the cyber-attack area. Conventional signature-based intrusion detection systems (IDS) will no longer be adequate for identifying advanced threats. A data-driven, adaptive approach that can identify malicious network activity is provided by machine learning (ML) techniques. This review aims to study, compare, and analyze ML-based approaches in IDS and improve the security defense mechanism. Methods: This systematic review followed the PRISMA 2020 guidelines. ML-based IDS peer-reviewed papers were identified from five scientific databases. Abstracts, full texts, and titles were filtered using predetermined inclusion and exclusion criteria, resulting in a sample of 53 primary studies. Data extraction included the algorithms used, the data used, and the metrics used to evaluate. Findings: The data show that most supervised ML techniques, such as decision trees, support vector machines, ensemble models, and deep learning systems (e.g., convolutional and recurrent neural networks), are predominant. In the majority of studies, high detection accuracy was obtained in controlled experimental settings. Conclusions: ML is a significant addition to intrusion detection, especially for anomaly detection and zero-day attack detection. However, the actual implementation is still limited due to the lack of detailed assessment systems and strict robustness testing. Future studies can focus on reproducibility, the use of diverse datasets, adversarial robustness, and the development of explainable ML methods.

D. Okeke · 0 citations
Review Open access Aug 2026

AI-ENHANCED INFORMATION SECURITY FRAMEWORKS FOR CLOUD-ENABLED IOT NETWORKS: A COMPREHENSIVE REVIEW

This review paper critically examines the integration of Artificial Intelligence (AI) and Machine Learning (ML) techniques to enhance information security within Cloud-IoT networks, focusing on hybrid Deep Learning models (CNN-LSTM), predictive analytics, and automated threat response mechanisms.

R. Saravanakumar, V.Anuratha, M.Elamparithi · 0 citations
Conference Jul 2026

A Comprehensive Review of AI-Powered Intrusion Detection Techniques in a Cloud Computing Environment

In Cloud Computing, Artificial Intelligence (AI)-driven intrusion detection focuses on identifying anomalies, unauthorized access, and malicious activities across dynamic cloud environments. Besides, the Intrusion Detection System (IDS) is also crucial in strengthening the overall cybersecurity defenses, thereby assisting institutions or organizations in detecting malicious activities to improve their security and preserve sensitive data. Conventional security approaches often fail to handle constantly evolving attack patterns in the cloud. Prevailing signature-based schemes do not have the ability to identify unknown threats, thereby generating false alarms. Besides, they require large labelled databases for adapting to changing workloads. This survey examines several intrusion detection approaches in the Cloud Computing Environment. The methods are categorized as Machine Learning (ML), Federated Learning (FL), Deep Learning (DL), and Big Databased models. Further, to provide a comprehensive assessment, 25 research papers on intrusion detection are collected and reviewed. Further, a general outline for detecting intrusions is explained, and then the literature review of each technique with its pros and cons is elaborated. The research gaps that are encountered by the existing techniques are also presented. In addition, this survey also highlights the analysis on the basis of various factors, like publication year, methodology, tools, indicators, and databases utilized.

P. Raja, J. Sathiamoorthy · 0 citations
Open access Aug 2026

Intelligent DDOS Attack Detection and Mitigation Using Machine Learning Techniques

Distributed Denial-of-Service (DDoS) attacks remain among the most disruptive threats to modern network infrastructure, with adversaries continually adapting their strategies to overwhelm cloud platforms, Internet-of-Things (IoT) deployments, and Software-Defined Network (SDN) environments. Traditional signature-based intrusion detection systems exhibit inherent inflexibility against novel attack vectors, motivating a shift toward intelligent, data-driven defense mechanisms. This paper presents an intelligent DDoS detection and mitigation framework that combines classical Machine Learning (ML) classifiers with Deep Learning (DL) architectures to achieve high-fidelity, low-latency attack identification across heterogeneous network topologies. Evaluated on the CICDDoS2019, NSL-KDD, and UNSW-NB15 benchmark datasets, the proposed hybrid framework incorporating XGBoost and a Bidirectional LSTM model achieves a classification accuracy of 99.31%, a precision of 99.18%, a recall of 99.27%, and an F1-score of 99.22%, outperforming standalone classifiers while sustaining sub-millisecond detection latency under realistic traffic loads. SDN-assisted rule insertion further reduces the mean mitigation response time to 8.4 ms. The results affirm the viability of deploying intelligent, explainable ML-based defense pipelines in production-grade network environments.

S. Singh, Alok Kumar · 0 citations