Adaptive malware detection under concept drift: an evolutionary deep learning framework with GA-based retraining

Abstract

The continuous evolution of malware can lead to performance degradation in static detection models; a phenomenon commonly associated with concept drift. This study presents an adaptive retraining approach that addresses concept drift while improving the performance of deep learning-based malware classifiers. The proposed methodology employs a convolutional neural network (CNN), selected for its ability to identify localized malicious patterns within application programming interface (API) call sequences. A key contribution of this work is the integration of a genetic algorithm (GA) into the proposed framework to generate new feature patterns for adaptive retraining. These GA-generated patterns enhance the CNN's ability to adapt to distributional shifts by providing an enriched feature set for model retraining. The experimental results demonstrate the effectiveness of the proposed approach. The static CNN initially achieved an accuracy of 98%; however, its performance declined by 12 percentage points, from 98% to 86%, when evaluated on a newly collected dataset, indicating the effect of concept drift. The proposed adaptive framework mitigated this performance degradation by incorporating GA-generated feature patterns during retraining, improving the accuracy to 97.7% on the same dataset. These findings demonstrate the potential of the proposed framework to provide robust and adaptive malware detection under evolving data distributions.