Skip to content
Open access

Deep Learning-Based Framework for Zero-Day Attack Detection in Encrypted Network Traffic Using Python

Jul 2026 · Al-Noor Journal of Engineering Management and Computer Science · 0 citations · 21 references

TL;DR

A deep learning-based intrusion detection framework that utilizes flow-level statistical features and a hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN–BiLSTM) architecture to capture both spatial and temporal characteristics of network traffic is proposed.

Abstract

The increasing adoption of encryption in modern communication networks has reduced the effectiveness of traditional intrusion detection systems that rely on packet payload inspection. Consequently, there is a growing demand for payload-agnostic intrusion detection approaches capable of analyzing network behavior without accessing packet contents. This study proposes a deep learning-based intrusion detection framework that utilizes flow-level statistical features and a hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN–BiLSTM) architecture to capture both spatial and temporal characteristics of network traffic. The framework was evaluated using the UNSW-NB15 and CIC-IDS2018 benchmark datasets under both in-dataset and cross-dataset evaluation settings. Experimental results demonstrated excellent classification performance in in-dataset experiments, achieving high accuracy and area under the curve (AUC) values. A threshold calibration strategy was further employed to reduce false-positive rates and improve operational reliability. However, cross-dataset evaluation revealed a considerable decline in detection performance, highlighting the impact of domain shift on model generalization across heterogeneous network environments. These findings demonstrate the effectiveness of flow-level statistical features for intrusion detection while emphasizing the importance of developing more robust learning strategies to improve generalization beyond single-dataset training.

Read PDF

Similar papers

Open access Aug 2026

Deep Learning-Based Network Intrusion Detection Using Hybrid CNN and LSTM Architecture

The findings indicate that hybrid deep learning techniques can improve network security by enhancing intrusion detection capability while reducing false alarms.

A. O. Jimoh-Mahmud, Abubakar Dayyabu, Abubakar Sadiq Idris et al. · 0 citations
Open access

Performance evaluation of deep learning models for intrusion detection using network traffic

This study examines a one-dimensional Convolutional Neural Network and a hybrid model, investigating how both architectures can detect network attacks in binary and multiclass classification settings, and provides actionable insights for practitioners choosing between deep learning and classical approaches under real-world NIDS deployment constraints.

Rachid Cheick Mohamed · 0 citations
Open access Jul 2026

Explainable attention-based intrusion detection for encrypted 5G network traffic

The widespread adoption of end-to-end encryption in 5G networks limits the effectiveness of traditional intrusion detection systems that rely on payload inspection. This challenge is particularly critical for detecting Advanced Persistent Threats (APTs), which employ low-rate, long-duration, and stealthy communication strategies to evade conventional defenses. This study presents a privacy-preserving intrusion detection framework that operates exclusively on flow-level traffic metadata without deep packet inspection. Network packets are aggregated into bidirectional flows, from which temporal, statistical, and directional features are extracted to characterize behavioral patterns. A Transformer-based model with self-attention is employed to capture long-range dependencies across encrypted traffic sequences and identify subtle, temporally dispersed attack indicators. The framework is evaluated on a large-scale 5G-relevant dataset containing over one million flow records and compared against classical machine learning, ensemble, CNN, and LSTM models. Results demonstrate high recall and strong F1-score in distinguishing APT from benign traffic. Attention-based and feature-level explanations further reveal that prolonged communication, irregular timing gaps, and directional asymmetry significantly influence detection decisions. The findings support the practicality of explainable Transformer models for secure and scalable APT detection in encrypted 5G environments.

Raghu Dhumpati, Varun Vemulapalli, Udayaraju Pamula et al. · 0 citations
Conference Open access 2025

Deep Learning–Based Real-Time Detection of Distributed Denial-of-Service Attacks in Network Traffic

: DDoS attacks have remained very troublesome in terms of the availability and reliability of online services through flooding network infrastructures with malicious traffic. Conventional signature-based and rule-based detection methods are not always useful to deal with the magnitude, variety and dynamism of these attacks. To overcome these problems, this piece of work proposes a deep learning-based real-time detection framework of DDoS attacks in network traffic. The suggested model combines both Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks in order to identify discriminative spatial characteristics of traffic patterns and sequence dependencies in the flow of packets, respectively. A hybrid CNN-LSTM model is trained and tested on a publicly available dataset on DDoS traffic and performance measured based on Accuracy, Precision, Recall, F1-score, and ROC-AUC. In order to have transparency, explainable AI (XAI) techniques like SHAP or LIME are used to explain the predictions of the model. To be deployed, the structure includes a Streamlit-based interface that enables real-time traffic simulation, live packet-level prediction, and detection result visualization. The results of the experiments indicate the usefulness of the offered method in attaining impressive detection and providing real-time monitoring of the DDoS threats, which can be applied practically.

D. K., M. K · 0 citations
Open access Aug 2026

A Comparative Evaluation of Deep Learning Architectures for Binary Network Intrusion Detection Using the NSL-KDD Dataset

The rapid growth of digital communication technologies, cloud computing, and Internet of Things (IoT) devices has increased both the frequency and sophistication of cyber-attacks, making effective intrusion detection an essential component of modern cybersecurity systems. Traditional signature-based intrusion detection systems (IDS) are effective against known attacks but fail to detect previously unseen or evolving threats. This study investigates the application of deep learning models for binary network intrusion detection using the NSL-KDD benchmark dataset. Three standalone architectures, Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) networks, and Deep Neural Networks (DNN), are implemented and evaluated, alongside a CNN-LSTM Hybrid model that integrates spatial and sequential learning, and a DNN-LSTM Ensemble model that combines independently trained DNN and LSTM predictions through weighted averaging. Following data cleaning, categorical encoding, normalization, and Random Forest-based feature selection (41 features reduced to 20), all models were trained and evaluated under identical conditions using Accuracy, Precision, Recall, F1-Score, ROC-AUC, training time, and inference time. The standalone DNN model achieved the best overall performance, with 80.98% accuracy, 97.08% precision, 68.66% recall, 80.43% F1-score, and 96.11% ROC-AUC, while also requiring the shortest training time (39.69 s). The CNNLSTM Hybrid model attained the highest precision (97.23%) but did not outperform the standalone architectures overall, and the DNN-LSTM Ensemble produced balanced but not superior results. These findings indicate that carefully designed standalone architectures can match or exceed the performance of more complex hybrid and ensemble models for binary intrusion detection, while incurring substantially lower computational cost. The study contributes a controlled, commonframework comparison of five deep learning architectures and provides practical guidance for selecting computationally efficient models for anomaly-based intrusion detection.

Ketki Naik, Sanjeev Ghosh · 0 citations
Open access Jul 2026

Hybrid Transformer–CNN–BiLSTM Model with Mahalanobis Embedding for Zero-Day Intrusion Detection

Abstract: The increasing complexity of cyber threats and the limitations of signature-based intrusion detection systems have made the detection of zero-day attacks a critical challenge in network security. Although machine learning and deep learning methods have achieved high accuracy for known attacks, their ability to generalize to previously unseen attack families remain limited. This paper proposes a hybrid intrusion detection model that integrates a Feature Tokenization-based Transformer with a CNN–BiLSTM model to capture global feature interactions and local sequential dependencies in tabular network traffic data. The learned fusion embedding is further exploited using a Mahalanobis distance–based anomaly detector to identify zero-day intrusions in an open-set setting. Experiments were conducted on the UNSW-NB15 dataset using a strict held-out attack family protocol, where Analysis, Backdoor, and Worms were excluded from training and reserved for zero-day testing. The proposed model achieved 94.64% accuracy, 0.9610 F1-score, and 0.9902 AUC on the known test set, that indicate the strong zero-day detection capability with high family-wise true positive rates for Backdoor and Worms. Five-fold cross-validation, ablation study, and statistical significance analysis further confirmed the robustness and effectiveness of the proposed framework. The results indicate that the integration of transformer-based feature learning, sequential modeling, and covariance-aware embedding analysis provides a reliable and extensible solution for both known attack classification and zero-day intrusion detection.

Deepali Hiraman Gavhane, Santosh Gaikwad, Chitra Desai · 0 citations