An Explainable Multi-Model Deep Learning Framework for Breast Cancer Diagnosis from Histopathological Images

Abstract

Breast cancer is a significant cause of cancer-related deaths among women worldwide. Its early identification and screening are essential for improved patient outcomes and reduced mortality rates. Histopathological image analysis is considered as the gold standard for the diagnosis and prognosis of breast cancer. Nevertheless, the complexity of Whole-Slide Images (WSI) and their manual examination make this task time consuming, and prone to pathologist subjectivity. Recently, Deep Learning (DL) technology has achieved remarkable success in computer vision. However, their application still faces critical challenges in pathology analysis, including Region-of-Interest (RoI) scale variations, inter- and intra-class heterogeneity, diverse staining protocols, and the scarcity of annotated datasets. Furthermore, DL model’s findings are opaque and lack decision-level transparency. This study proposes a novel explainable multi-model DL framework for breast cancer classification leveraging histopathological images. The framework integrates Contrast Limited Adaptive Histogram Equalization (CLAHE) for image contrast enhancement, and diverse data augmentation to mitigate class imbalance and overfitting. Proposed architecture ensembles two branches, one employes DenseNet201 benefiting from Transfer Learning (TL) via ImageNet weights, while other utilizes a custom light weight attention based Hierarchal Feature Fusion (HFF) Network. DenseNet201 utilizes multilevel features to effectively tackle gradient vanishing issues and capture intricate feature representations, while HFF-Net, designed specifically for biomedical imaging, leverages HFF stem and multiscale feature extraction with Swish activation to enhance learning stability. Attention mechanism introduced within HFF-Net further refines the output features. Final feature vectors from the two branches are fused at the Global Average Pooling (GAP) layer, consolidating discriminative information. Experimental results on BRACS dataset demonstrate the proposed framework achieves 97.15% accuracy, 92.59% precision, 93.81% recall, and a 93.19% F1-score in screening tasks, while for grading tasks, it attains 84.08% accuracy, 83.39% precision, 83.64% recall, and 83.44% F1-score on 4391 test samples. Additionally, Gradient-Class Activation Mapping (Grad-CAM) saliency heatmap are generated for visual representation of proposed model’s choices, thereby increased transparency. The integration of these advanced techniques significantly enhances diagnostic reliability, addressing the challenges in histopathological image analysis.