Skip to content
Open access

AI-Based Adaptive Malware Detection Using Portable Executable (PE) Header Analysis and Hybrid Machine Learning

Jul 2026 · International journal of computer information systems and industrial management applications · 0 citations

TL;DR

Experimental results demonstrate that the proposed approach achieves high classification performance while improving transparency in malware detection decisions, making it suitable for practical cybersecurity applications.

Abstract

With the rapid evolution of malware and sophisticated evasion techniques used, the efficiency of traditional detection approaches will be increasingly compromised since malware creators frequently leverage polymorphic, metamorphic, packing, encryption, and code obfuscation techniques to circumvent traditional security methods. This work includes an AI-based adaptive malware detection system that utilizes the Portable Executable (PE) headers along with hybrid machine learning models. The system conducts static analysis of malware without the need to run malware in sandboxed environments to gather information regarding the structure of executable files. In total, the system trains and tests three machine learning models - Random Forest, eXtreme Gradient Boosting (XGBoost), and Artificial Neural Network (ANN). Training and testing are conducted on a dataset of 19,611 executable files described with 79 characteristics extracted from Portable Executable headers. To enhance predictive accuracy, a hybrid ensemble model is used, while SHapley Additive exPlanations (SHAP) enable predictability. The experiments conducted yielded an accuracy of 99.13% and an Area Under the Curve (AUC) value of 0.9984 with an extremely low false negative rate. The proposed framework combines a hybrid ensemble of Random Forest and XGBoost classifiers with SHapley Additive exPlanations (SHAP) to provide accurate and interpretable malware detection using static PE-header analysis. Experimental results demonstrate that the proposed approach achieves high classification performance while improving transparency in malware detection decisions, making it suitable for practical cybersecurity applications.

Read PDF

Similar papers

Conference Open access 2026

A Comparative Study of Machine Learning Algorithms for Real-time Detection of Windows Portable Executable (PE) Malware

Evaluated tree-based ensemble models, Random Forest, XGBoost, and LightGBM, for static malware detection in PE files show that tree-based ensembles outperform deep learning models, such as Multilayer Perceptrons (MLPs) and Convolutional Neural Networks (CNNs), as well as traditional machine learning approaches in handling high-dimensional tabular data.

Aleksandar Sandro Cvetković, S. Adamovic, Marko Šarac · 0 citations
Open access 2026

Explainable AI for Malware Detection: Feature Importance Analysis Using SHAP and Machine Learning

A machine learning-based malware detection framework using features extracted from Windows Portable Executable (PE) files is presented and the SHAP-based feature importance approach helps identify the most influential features responsible for malware detection while also reducing model complexity and computational cost.

Balachandra Chikkoppa, H. J. · 0 citations
Open access 2026

Large-Scale Static Malware Detection Using Classical Machine Learning Models: An Evaluation on the EMBER Dataset

The results suggest that well-designed classical ensemble approaches are still quite competitive for scalable and interpretable static malware detection even with the rising usage of more and more powerful machine learning architectures.

A. Fauzan, Tito Pinandita, Aulia Desy Nur Utomo · 0 citations
Conference Jul 2026

A Comparative Analysis of Machine Learning and Deep Learning Methods for Malware Detection in Portable Executable Files

Malware severely threatens cybersecurity through Portable Executable (PE) files. In this study, the performances of machine learning and deep learning models for malware detection are compared using static analysis features extracted from PE files. Tree-based ensemble models like Random Forest, XGBoost, LightGBM, and CatBoost, along with MLP, DNN, and 1D-CNN deep learning architectures, were trained on a dataset comprising 62,485 samples and 15 features. Experimental results proved that ensemble algorithms outperform deep learning models on low-dimensional tabular data. Among the models, Random Forest achieved the highest success with 99.65% accuracy and a 99.59% F1-score. This academic study demonstrates that lightweight and high-performance tree-based models offer a more efficient solution for malware detection compared to computationally expensive deep neural networks.

Gokay Mutlu, Neşe Rihani, E. Bayazit · 0 citations
Review Open access Aug 2026

A Comparative Evaluation of Malware Families and Machine-Learning Detection Techniques, and an Optimized Stacked-Ensemble Model for Predicting Software Maliciousness

Malware is growing fast in volume, variety, and sophistication, and traditional signature-based defences can no longer keep up. This has driven a shift toward machine-learning (ML) based detection. This paper has two main goals. First, it compares the major malware families - viruses, worms, trojans, ransomware, spyware, adware, rootkits, botnet clients, and polymorphic/metamorphic variants - looking at how each spreads and what damage it causes. It also surveys the intelligent algorithms behind modern ML-based malware detection (probabilistic, ensemble, kernel-based, and deep-sequential models), and works on three gaps in the existing literature: a disconnect between surveys and reproducible benchmarks, limited joint attention to accuracy, efficiency, and robustness together, and under-explored heterogeneous stacking for static malware detection. Second, building on this analysis, the paper designs and tests an efficient stacked-ensemble model that estimates how likely a given executable is to be malicious. Using the ClaMP static PE-header benchmark (5,184 labelled Windows executables, 55 raw features), ten baseline classifiers are benchmarked first. An embedded feature-selection step (Random Forest Gini importance) then cuts the feature space by about 49% before training a four-member ensemble - Random Forest, XGBoost, a Multilayer Perceptron, and Gradient Boosting - combined through a logistic-regression meta-learner.  Proposed model in this research performs 98.14% accuracy, 98.20% F1 score and an AUC of 0.998 on given data, it uses five fold cross validation accuracy, our results are compared against recent malware research/types are identified for future research work/area.

Deepak Singh Rana, Sushil Chandra Dimri · 0 citations
Open access Jul 2026

Intelligent Android Malware Classification Using Equilibrium Optimizer and Deep LearningModel

An intelligent Android malware detection framework that combines deep learning with the Equilibrium Optimizer to improve detection performance is presented, providing an effective and reliable solution for securing Android devices against evolving malware threats.

Aishwarya Eklar, G.Rajini · 0 citations