HEDM-KD: A Parallel Heterogeneous Knowledge Distillation Framework for Class-Imbalanced Intrusion Detection

Abstract

Network intrusion detection is critical for system security, particularly the accurate identification of rare and irregular attack samples. Class imbalance remains a major challenge, as the predominance of normal traffic can bias decision boundaries toward majority classes and degrade minority-attack detection. Traditional knowledge distillation approaches rely on large pre-trained teacher models, which incur high computational costs and offer limited adaptability to dynamic uncertainty differences between teacher and student models under imbalanced conditions. To address these issues, we propose HEDM-KD, a parallel heterogeneous ensemble knowledge distillation framework based on CNNs. The framework first constructs multiple heterogeneous subsets via bootstrap sampling and applies diverse imbalance-handling strategies to enhance data diversity. A dynamic mutual learning mechanism then enables synchronous training across branches: in each training round, the branch with the highest validation F1-score is selected as the teacher, and its convolutional weights are transferred to the remaining student branches. An entropy-driven dynamic weighting strategy further adaptively balances the hard-label loss and distillation loss according to model output uncertainty, improving robustness under imbalanced distributions. Finally, multi-branch hard voting enhances classification stability. Extensive experiments on CarHacking, CIC-IoV2024, UNSW-NB15, and CICIDS2017 show that HEDM-KD achieves competitive performance against single resampling methods, conventional knowledge distillation variants, and models including AdaBoost, ResNet18, TFTKD, and DFF-DA in terms of accuracy, precision, recall, and F1-score. Notably, it maintains strong detection capability on the highly imbalanced UNSW-NB15 dataset while attaining near-perfect metrics on CarHacking, CIC-IoV2024, and CICIDS2017. With its lightweight design, HEDM-KD effectively mitigates the impact of class imbalance and demonstrates potential for improving detection robustness in resource-constrained scenarios.