Classifier-Dependent Benefits of Pseudo-Labeling for Semi-Supervised Android Malware Attribution

Abstract

Detecting and classifying Android malware families remains challenging due to high feature dimensionality, class imbalance, and the high cost of expert-labeled data. Semi-supervised learning (SSL) offers a way to leverage unlabeled samples, but prior works rarely test whether SSL benefits generalize across classifier types or report statistical significance. We present a systematic evaluation of pseudo-labeling across six classifiers (LightGBM, XGBoost, Random Forest, Logistic Regression, MLP, and SVM) on the CICMalDroid 2020 dataset, using five-fold stratified cross-validation and paired t-tests across five labeled ratios (1-20%). We find that SSL benefit is strongly classifier-dependent: SVM shows the largest significant gain (+4.4% accuracy at 5% labels, p = 0.0028), LightGBM improves modestly (+0.8 to +1.3% at 2-5% labels), while Random Forest is significantly harmed at low label ratios (-3.1% at 1% labels). Per-class analysis reveals SSL disproportionately benefits the hardest-to-classify families, with Adware F1 improving by +13.8 percentage points versus only +0.8 for the already well-classified Benign class. We further show that approximately 800 labeled samples (10% of the dataset) yield near-optimal performance across all classifiers. These findings offer practical guidance on when and with which classifier pseudo-labeling is worthwhile for Android malware classification.