Skip to content
Open access

Machine Learning-Based Malware Detection: A Comparative Study of Random Forest, Decision Tree, KNN, and Linear SVM

Aug 2026 · American Journal of Agricultural Science, Engineering and Technology · 0 citations · 15 references

TL;DR

Random Forest achieves the highest performance with a test accuracy of 96.3%, F1-score of 0.947, and AUC of 0.993, establishing it as the optimal algorithm for static malware detection tasks and establishing it as the optimal algorithm for static malware detection tasks.

Abstract

The growing prevalence of malware presents a critical threat to cybersecurity, causing substantial financial and operational damage to organizations worldwide. Traditional signature-based detection approaches are increasingly insufficient against polymorphic and zero-day threats. This paper presents a comprehensive comparative study of four machine learning (ML) algorithms — Random Forest (RF), Decision Tree (DT), K-Nearest Neighbor (KNN), and Linear Support Vector Machine (SVM) — for malware detection using static feature analysis on Portable Executable (PE) files. Experiments were conducted on a combined dataset derived from Drebin-215 and Malgenome-215 containing 18,830 instances with 208 features. A stratified 10-fold cross-validation with GridSearch CV hyperparameter tuning was employed. Evaluation metrics include accuracy, precision, recall, F1-score, and Area Under the ROC Curve (AUC). Results demonstrate that Random Forest achieves the highest performance with a test accuracy of 96.3%, F1-score of 0.947, and AUC of 0.993, outperforming all other classifiers and establishing it as the optimal algorithm for static malware detection tasks.

Read PDF

Similar papers

Open access 2026

Large-Scale Static Malware Detection Using Classical Machine Learning Models: An Evaluation on the EMBER Dataset

The results suggest that well-designed classical ensemble approaches are still quite competitive for scalable and interpretable static malware detection even with the rising usage of more and more powerful machine learning architectures.

A. Fauzan, Tito Pinandita, Aulia Desy Nur Utomo · 0 citations
Conference Jul 2026

A Comparative Analysis of Machine Learning and Deep Learning Methods for Malware Detection in Portable Executable Files

Malware severely threatens cybersecurity through Portable Executable (PE) files. In this study, the performances of machine learning and deep learning models for malware detection are compared using static analysis features extracted from PE files. Tree-based ensemble models like Random Forest, XGBoost, LightGBM, and CatBoost, along with MLP, DNN, and 1D-CNN deep learning architectures, were trained on a dataset comprising 62,485 samples and 15 features. Experimental results proved that ensemble algorithms outperform deep learning models on low-dimensional tabular data. Among the models, Random Forest achieved the highest success with 99.65% accuracy and a 99.59% F1-score. This academic study demonstrates that lightweight and high-performance tree-based models offer a more efficient solution for malware detection compared to computationally expensive deep neural networks.

Gokay Mutlu, Neşe Rihani, E. Bayazit · 0 citations
Open access 2026

Explainable AI for Malware Detection: Feature Importance Analysis Using SHAP and Machine Learning

A machine learning-based malware detection framework using features extracted from Windows Portable Executable (PE) files is presented and the SHAP-based feature importance approach helps identify the most influential features responsible for malware detection while also reducing model complexity and computational cost.

Balachandra Chikkoppa, H. J. · 0 citations
Open access Jul 2026

AI-Based Adaptive Malware Detection Using Portable Executable (PE) Header Analysis and Hybrid Machine Learning

Experimental results demonstrate that the proposed approach achieves high classification performance while improving transparency in malware detection decisions, making it suitable for practical cybersecurity applications.

V. Padmapriya, S Uma, S Sumathi et al. · 0 citations
Review Open access Aug 2026

A Comparative Evaluation of Malware Families and Machine-Learning Detection Techniques, and an Optimized Stacked-Ensemble Model for Predicting Software Maliciousness

Malware is growing fast in volume, variety, and sophistication, and traditional signature-based defences can no longer keep up. This has driven a shift toward machine-learning (ML) based detection. This paper has two main goals. First, it compares the major malware families - viruses, worms, trojans, ransomware, spyware, adware, rootkits, botnet clients, and polymorphic/metamorphic variants - looking at how each spreads and what damage it causes. It also surveys the intelligent algorithms behind modern ML-based malware detection (probabilistic, ensemble, kernel-based, and deep-sequential models), and works on three gaps in the existing literature: a disconnect between surveys and reproducible benchmarks, limited joint attention to accuracy, efficiency, and robustness together, and under-explored heterogeneous stacking for static malware detection. Second, building on this analysis, the paper designs and tests an efficient stacked-ensemble model that estimates how likely a given executable is to be malicious. Using the ClaMP static PE-header benchmark (5,184 labelled Windows executables, 55 raw features), ten baseline classifiers are benchmarked first. An embedded feature-selection step (Random Forest Gini importance) then cuts the feature space by about 49% before training a four-member ensemble - Random Forest, XGBoost, a Multilayer Perceptron, and Gradient Boosting - combined through a logistic-regression meta-learner.  Proposed model in this research performs 98.14% accuracy, 98.20% F1 score and an AUC of 0.998 on given data, it uses five fold cross validation accuracy, our results are compared against recent malware research/types are identified for future research work/area.

Deepak Singh Rana, Sushil Chandra Dimri · 0 citations
Open access Jul 2026

A Unified Machine Learning-Based IDS/IPS Framework with Bio-Inspired Feature Selection for Real-Time Detection of Malware-Laden URLs

Cybersecurity threats have become increasingly sophisticated and dynamic due to the rapid evolution of malicious software and attack techniques. However, the reliance of traditional IDS/IPS implementations on signature-based detection limits their ability to identify novel, rapidly evolving, and adaptive malware, despite their effectiveness against known cyber threats. This paper presents a unified machine-learning framework for defence against malware-laden URLs, which simultaneously targets intrusion detection (accuracy first, out of band) and intrusion prevention (latency first, inline) through module-aware, bio-inspired feature selection. The detection module employs random forest (RF) with the bat algorithm (BA), achieving 99.52% accuracy under stratified fivefold validation. This approach yields thousands of additional correct decisions per million URLs and reduces false negatives and false positives without adverse throughput effects. The prevention module utilizes a decision tree (DT) with the HHO ∩ BA consensus subset, achieving an inference time of 1 ms per URL with competitive accuracy (98.96%), enabling more than 1,000 URLs per second per core and meeting strict inline decision budgets. Standard metrics, such as accuracy, precision, recall, and F1-score, along with timing analyses, confirm that aligning feature selection with module objectives strikes a practical balance between detection quality and latency. The resulting guidance is direct: employ RF with BA for intrusion detection when optimal discriminative performance is essential, and adopt DT with the HHO ∩ BA subset for intrusion prevention when millisecond-scale latency dictates deployment.

Mohammad Sh. Daoud, Mosleh M. Abualhaj, Sumaya S. Al-Khatib et al. · 0 citations