AT-IDS: An Adaptive Transformer-Based Intrusion Detection System with Dynamic Feature Learning and Dual-Path Attention

Abstract

The problem of intrusion detection in networks is of major concern due to an increase in the number of sophisticated attack patterns. Typical machine learning models are limited either in terms of feature importance ranking capability or their ability to adapt to ever-changing traffic patterns in high-dimensional and class imbalanced traffic distributions (benign:attack $\approx 6.8: 1)$ of real-world scenarios. In this study, the Adaptive Transformer-based Intrusion Detection System (AT-IDS) is proposed which is developed on the comprehensive data set UNSW-NB15 of 2,540,044 samples. AT-IDS employs five prominent technologies which work in conjunction with each other and include: Dynamic Feature Importance Learning Network (DFILN) to weight the input features in an end-to-end manner using a gated residual mechanism,Dual-Path Adaptive Attention Calibration (DAAC) to control self-attention in local sliding-window view as well as global receptive field, Context Preservation Fusion (CPF) to connect shallow tokenized representation with deeper Transformer representation, Adaptive Hybrid Pooling which helps to combine classification (CLS) token, mean and max aggregations; and a trainable Adaptive Focal Loss model with backpropagation capability of tuning its focusing exponent $\gamma$. In a stratified experiment with train-validation-testing ratios of 70/15/15, AT-IDS achieves an accuracy score of 99.02%, a precision score of 95.67%, and a recall score of 97.79% with an F1 score of 96. 71% on the withheld testing set of 381,008 samples, thus performing better than other methods, namely CNN-LSTM, TabNet, and BERT4IDS (McNemar test, $p<0.001)$. The five-fold cross-validation results (on the training set) confirm the results, with the mean F1-score of 96.71%. With a Shapley Additive Explanations (SHAP) analysis of DFILN gate scores and DAAC per head attention calibration confirming the right decision-making processes of the algorithm based on what is already known in the field of network security. The cross dataset validation has been run on the NSL-KDD (accuracy of 98.12%) confirming the generalizability of the results. The results of ablation studies prove the significance of each of the techniques used. The overall implementation and demonstration of the findings have been reported via final full-stack application via a full-stack application using the Flask REST API with performance of median end-to-end latency of 38 ms with every batch consisting of 512 flows.