A Novel Hybrid Deep Learning Architecture Combining CNNs, Vision Transformers, and Multi-Scale Attention for Enhanced Detection of Breast Cancer in Histopathology Images

Abstract

Breast cancer has been one of the major causes of cancer mortality in the world. According to WHO report 2.5 million deaths are predicted as a result of breast cancer in the world in 2040. Although deep learning has demonstrated encouraging histopathology image analysis, the current methods frequently fail to provide local morphological information and global contextual information at the same time. In this paper, we present our HrybridViT-CAM, a hybrid deep learning system that integrates convolution neural networks( CNN ), Vision Transformers( ViT ) and multi-scale attention system in order to classify breast cancer using histopathology images. Its architecture has a two-way structure: a CNN arm (EfficientNetB7) to extract local features and a vision transformer arm to analyze the global context, fused together with a cross-attention fusion block. We use convolution block attention modules(CBAM) and deformal attention to improve feature discrimination. The model was tested on the BreaKHis dataset with various magnifications (40x, 100x, 200x, 400x) in binary as well as in multi-class classification. For binary classification (benign vs. malignant), HybridViT-CAM achieved accuracies of 99.87%, 99.42%, 98.95%, and 98.31% at 40χ, 100χ, 200x, and 400x magnifications, respectively. For eight-class subtype classification, the corresponding accuracies were 98.76%, 97.89%, 97.23%, and 96.54%, respectively. Grad-CAM++ and attention visualization techniques allowed explaining the results, which showed high correspondence(93.7% agreement) with pathologist diagnostic criterial. The proposed model was able to detect the malignant regions of interest (ROIs) like nuclei pleomorphism, atypia chromatin patterns and architectural distortions which are comparable to clinical diagnostic standards.