Obfuscated Malware Detection Through Ensemble Learning
Modern malwares utilize different obfuscation techniques to hide their behaviors and overcome traditional signature-based detection methods. This paper investigates the use of machine learning techniques to detect obfuscated malware in the Windows operating system trained on memory-based features. This research considers different malware types and analyzes several obfuscation techniques used by malwares to bypass antivirus detection. An ensemble machine learning model is developed based on four algorithms: SVM, Random Forest, Gradient Boosting, and CatBoost. Voting is used for classification based on results of each model. The ensemble model is evaluated using a dataset of obfuscated malware samples, namely CIC-MalMem-2022, and validated through cross-validation. Experimental results show high detection rate where the accuracy and F1-score are 99.99% and 99%, respectively. The results show that machine learning can enhance malware detection against obfuscated malware threats.