Skip to content

AI-Based Malware Behavior Classification and Mitigation System Using Machine Learning

2026 · Research & Review: Machine Learning and Cloud Computing · 0 citations

TL;DR

This study proposes an advanced, integrated behavior-based malware classification and automated mitigation framework specifically engineered for Windows operating environments, and introduces a multi-class risk-scoring mechanism to streamline Security Operations Center (SOC) incident triage.

Abstract

The continuous evolution and technical sophistication of modern cyber threats present severe challenges to enterprise networks, endpoint devices, and personal data privacy. Conventional security measures that rely predominantly on static file signatures and hardcoded indicators of compromise frequently fail against advanced polymorphic, packed, and fileless malware strains that dynamically alter their binary structures upon replication. To overcome these critical defensive limitations, this study proposes an advanced, integrated behavior-based malware classification and automated mitigation framework specifically engineered for Windows operating environments. Instead of examining static file hashes on disk prior to execution, the proposed system captures and analyzes high-velocity, multi-variable runtime telemetry in real time. This comprehensive telemetry monitoring encompasses low-level process creation paths, parent-child execution hierarchies, CPU and memory resource utilization percentages, file system modification counts, active network socket connections, and system event triggers. These dynamic behavioral attributes are processed through optimized supervised machine learning algorithms, specifically Decision Trees, Random Forests, Support Vector Machines, and Logistic Regression, to accurately classify software execution patterns as benign or malicious while predicting specific threat types. Furthermore, the framework introduces a multi-class risk-scoring mechanism (categorizing threats into Low, Medium, High, and Critical tiers) to streamline Security Operations Center (SOC) incident triage. Upon identifying malicious behavior exceeding predefined risk thresholds, an automated mitigation pipeline initiates immediate defensive protocols, including real-time security alerts and the instantaneous termination of hazardous processes. All monitored telemetry streams, classification outputs, risk evaluations, and mitigation actions are dynamically rendered through a responsive, interactive React.js web dashboard. Comprehensive experimental benchmarking reveals that the Random Forest model achieves the highest classification accuracy at 89.2%, outperforming alternative algorithms and demonstrating strong efficacy for proactive endpoint protection and automated threat containment.

View source

Similar papers

Review Oct 2026

Malware Analysis Using Data Mining Techniques: A Review of Static, Dynamic and Hybrid Analysis

Malware continues to evolve through obfuscation, packing, polymorphism, and other evasion techniques, challenging traditional detection approaches. Data mining (DM) and machine learning (ML) is become important by the day for identifying malicious software through static characteristics, runtime behaviour, and combinat...

Saud Adam Abdulkadir, Nor Fazlida Mohd Sani, Aziah Asmawi et al. · 0 citations
Review 2026

MAIR-Suite: An Automated Multi-Vector Malware Analysis, Threat Classification, and Incident Response Framework

Malware investigation often requires evidence from several sources, including file characteristics, signature matches, runtime behavior, and volatile memory, yet these findings are commonly produced by separate tools and are difficult to correlate. This paper proposes MAIR-Suite, an automated multi-vector malware analy...

Navya S. T., S. M. · 0 citations
Open access Sep 2026

Base Semantics—A Novel Approach for Minimizing Malware Detection Rules

This work provides a robust and scalable semantic‐based detection framework for both host‐based and IoT security systems in an evolving malware landscape by exploiting persistent semantic characteristics that remain invariant across evolving variants within malware families.

Khizar Hayat, Sadaf Hina, Fabiha Hashmat et al. · 0 citations
Open access Sep 2026

A Feature-Based Machine Learning Ensemble Model for Enhanced Malware Detection in Digital Banking Security

The rapid expansion of digital banking has introduced significant cybersecurity vulnerabilities, particularly from sophisticated malware threats that exploit polymorphic and zero-day evasion techniques. Traditional signature-based defenses demonstrate critical inadequacy in detecting novel threats, creating exploita...

Jennifer Ibimina Princewill · 0 citations
Open access Sep 2026

AI-Driven Malware Detection: An Entropy-Weighted Ensemble Framework for Static Analysis of Executable Files

A light-weight entropy-weighted ensemble detection approach for static malware detection that combines the predictions of Random Forest, Support Vector Machine and Gradient Boosting classifiers with weights proportional to the validation accuracy of each of these classifiers.

Kireet Muppavaram, Sudeepthi Govathoti, S. Rahaman et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.