ENHANCING INTRUSION DETECTION SYSTEMS WITH AI-BASED NETWORK TRAFFIC ANALYSIS: A FOCUS ON ANOMALY DETECTION
TL;DR
An intelligent approach based on event log analysis and machine learning methods is proposed to identify signs of potential information leaks in intrusion detection systems and demonstrates the advantages of regularized linear models.
Abstract
In an era of sophisticated cyber threats and the growing complexity of network infrastructures, intrusion detection systems (IDS) play a critical role in ensuring the security of digital assets. Traditional IDS methods based on predefined rules and signatures are becoming less effective against sophisticated attacks. The study covers the stages of data collection, preprocessing, model training, and evaluation. This study examines current security solutions, such as next-generation firewalls (NGFWs) and intrusion detection and prevention systems (IDS/IPS). In the experimental study, linear regression, Ridge regression, Lasso regression and Random Forest models were used. Particular attention is paid to the methodological problem of data leakage. Special attention was paid to the methodological problem of data leakage, and a leakage-free modeling scenario based only on previously known variables was created. Taking into account the pronounced right-hand asymmetry and heavy-tailed distribution of the dependent variable, the effect of using the log1p transformation on the generalization ability of the models was investigated. The obtained results demonstrate the advantages of regularized linear models. The use of log1p transformation in combination with Ridge regression allowed us to improve the quality of forecasting, reduce the variance of errors and increase the value of the determination coefficient R². Additional validation based on time-based data partitioning confirmed the methodological correctness of the conclusions drawn. The Random Forest model did not show any improvement in forecasting accuracy, but was used to analyze the structure of factors influencing the engagement rate. In addition, issues such as model interpretability, false positives, and the need for continuous learning are addressed. An intelligent approach based on event log analysis and machine learning methods is proposed to identify signs of potential information leaks.