Hybrid CNN-Transformer Based Brain Tumor Detection and Classification Using MRI: A Novel Framework with Multi-Level Attention and Regional Explainability
TL;DR
A novel hybrid convolutional neural networks and transformer architecture, HybCT-Net, augmented with a multi-level attention module and a regional explainability pipeline for brain tumor detection and classification is proposed, demonstrating superior performance than contemporary CNN, transformer and hybrid baselines.
Abstract
Automated and accurate classification of brain tumors from MRI (magnetic resonance imaging) for clinical applications is essential. However, it still poses a challenge owing to multiple factors. This includes the high intra-class heterogeneity, vague tumor boundaries, and lack of transparency in diagnostic reasoning. In this paper, we propose a novel hybrid convolutional neural networks and transformer architecture, HybCT-Net, augmented with a multi-level attention module and a regional explainability pipeline for brain tumor detection and classification. The framework employs local feature extraction of deep CNN encoders and the long-range dependency modeling capacity of lightweight vision transformers in conjunction. The MLAM incorporates attention mechanisms such as channel-wise squeeze-and-excitation gating, spatial convolutional block attention, and patch-based multi-head self-attention to enhance salient features at multiple semantic scales. A Hybrid Feature Fusion (HFF) is a block for adaptive fusion of the CNN feature maps and the transformer tokens that bridges the semantic gap between the convolution-based and attention-based features [24]. Moreover, the REP combines Gradient-weighted Class Activation Mapping with transformer attention rollout maps for producing regional heatmaps at pixel-wise spatial detail, enhancing clinical trust. The efficacy of the proposed model is established through extensive experiments on a multi-class brain MRI dataset with glioma, meningioma, pituitary tumor and no tumor. HybCT-Net attains a classification accuracy of 98.78%, with a macro-F1 score of 98.70% and an AUC of 0.9943. Comparative experiments demonstrate superior performance than contemporary CNN, transformer and hybrid baselines. The contributions of various architectural components are validated using ablation studies and computational analysis shows deployment feasibility. Qualitative visualizations suggest that the regional explainability maps correlate well with boundaries of the tumor as annotated by radiologists. Thus, the framework has the potential for use in clinical decision support in the real world.