Adaptive dual backbone feature fusion with compact latent representation learning for pap smear image classification

Abstract

Cervical cancer is highly preventable when precancerous cytological abnormalities are identified early; however, manual Pap smear assessment remains labour-intensive, time-consuming, and prone to inter-observer variability. Automated cervical cytology is also challenging because Pap smear cell images often exhibit subtle inter-class differences, considerable intra-class variation, and staining-related appearance changes. This study presents a compact dual-backbone framework for cell-level Pap smear image classification. ResNet-50 and ConvNeXt-Base are used to extract complementary morphological and texture-oriented representations, which are integrated through an input-conditioned fusion module that estimates image-dependent contributions from the two feature streams. To improve efficiency, a Hierarchical Autoencoder compresses the fused descriptor into a compact latent representation before final classification with a lightweight classifier. The framework is evaluated on the SIPaKMeD and Herlev cervical cytology datasets under binary and multiclass settings using five-fold cross-validation. It achieves macro-F1 scores of 99.66% and 98.15%, and binary F1-scores of 99.65% and 99.41% on SIPaKMeD and Herlev, respectively. Ablation studies indicate that the proposed fusion strategy outperforms the evaluated fixed- and learnable-fusion baselines under the same protocol, with McNemar’s test supporting statistically significant improvements over the selected baselines. Grad-CAM provides qualitative insight into stream-level activation patterns, while cross-dataset experiments offer an initial assessment of transferability under dataset shift. Overall, the findings support compact input-conditioned dual-backbone fusion as an effective approach for benchmark-level Pap smear image classification, while larger multi-centre and slide-level validation remains necessary before clinical deployment.