Multi-Modal Medical Image Fusion Using Hybrid CNN-Transformer Models for Early Detection of Chronic Diseases

An Attention-Enhanced CNN–Transformer Model for Brain Tumor Identification from MRI Scan

Magnetic resonance imaging (MRI) is very important for clinical diagnosis and treatment planning, because it can accurately detect and classify brain tumors. Traditional Convolutional Neural Network (CNN) models can extract local features, but they are not able to get long-range contextual information from complex medical images. The proposed system is to develop a CNN-Transformer based framework for automatic brain tumor detection and multi-class classification. The model combines Channel Attention Networks (CAN) with Vision Transformer based models (ViT)/Dual Vision Transformer(DaViT). Due to which the CAN enhances feature learning by spotlighting tumor-related channels. Also the model is validated on BraTS MRI dataset with four classes, i.e. pituitary tumor, meningioma, glioma and normal cases. The results obtained based on CNN-Transformer gives a higher accuracy of 97.6% and higher precision of 0.96 that enables the transformer module to capture global spatial relationships within the image.

Accurate Brain Tumor Classification Using MRI Images Based on A Hybrid Vision Transformer and BiLSTM Framework

Accurate classification of brain tumors using magnetic resonance imaging (MRI) is essential to clinical diagnosis and treatment. Nevertheless, the wide diversity in a single type of disease and high similarity between the tumors in different categories pose considerable challenges for deep learning models due to the characteristics of CNNs that are mainly for local features extracted, the necessity of reducing these limitations and constraints. This paper discusses an innovative hybrid deep learning paradigm in which an image is modeled by means of a vision transformer (ViT) and a Bi-directional long-term memory network (BiLSTM), resulting in an effective brain tumor classification. The application is based on the framework of ViT, capable of modeling overall context to extract high-level distinguishing features of MRI images, and the BiLSTM successfully capturing sequential dependencies inside of the extracted feature representations. This hybrid architecture is able to be very rich in modelling the spatial and contextual relationships that come with complex medical images. Results show that ViT–BiLSTM's classification performance is superior to those of traditional deep learning methods: among all the tumor categories its accuracy is higher, its fine-tuning more perfect, as well as, its Recall rates greater. This study demonstrates the efficacy of transformer-based hybrid architectures for medical image analysis with the proposal that by integrating a holistic attention framework with sequential modeling, they can yield substantially better patient diagnosing outcomes. The presented model is not only a viable recommendation for computer assisted diagnostic systems, but is also likely to help with clinical decision making in healthcare fields. The experimental results indicate that the proposed framework maintains a strong balance between accuracy and recall. Specifically, the model achieved accuracy/recall values of 92.5%/92.0% for gliomas, 91.2%/90.7% for meningiomas, and 93.5%/94.0% for brain tumors, resulting in high and consistent F1 scores across all categories.

Supervised Deep Learning Multi-Model ensemble of Computer Vision Transformer for Brain Tumor Detection

It is well known fact that in medical area, identifying the brain tumors properly through Magnetic Resonance Imaging (MRI) scans becomes essential for curing them within early time-frame and planning effectively for their treatments. While convolutional neural networks (CNNs) are popular in their work for finding local details in images, but, till now, they often have shortcomings on capturing broader context across the whole scan. New technology of Transformers, like the Swin model, can help eradicating these problems in understanding these global relationships but it also has some performance problem, when we use it standalone. To get rid of both issues, our study proposes SXM-Net, which is a hybrid deep ensemble model that utilizes a Swin Transformer model with CNN models especially Xception and MobileNetV2. Our approach setup classifies brain tumors into four types: glioma, meningioma, pituitary tumors, and normal (healthy brain with no tumor) tissue. Our proposed ensemble model expresses the superior experimental performance outcome in two different aspects: first one as compared to individual convolutional network model like Xception and mobileNetV2 seperately, and second one when we have hybrid convolutional networks like hybrid of Xception and mobileNetV2. But if we compare our ensemble model with individual swin transformer, then also our proposed model gives better result. Experiments were conducted on a dataset containing MRI images with multi-class tumors. The proposed ensemble SXM-Net for popular multi- categories of brain tumors achieved better test accuracy, with high average sensitivity values and high average specificity values, giving better experimental specific outcomes than individual traditional models.

A Hybrid CNN–Transformer Framework with Wavelet-Based Feature Extraction for Multimodal Cancer Detection

Finding cancer early and making a good treatment plan are both important for boosting survival rates. MRI, PET, and CT are advanced imaging techniques that have greatly improved cancer screening, staging, and therapy monitoring. However, their high costs and need for specialised equipment make them hard to get, especially in places with few resources. In this setting, optical imaging technologies are becoming affordable and portable options for finding cancer early. Image preprocessing techniques were used to improve data quality in order to deal with problems including dataset imbalance and noise. They used the Haar wavelet approach to extract features from medical photos that were important. A hybrid CNN-Transformer architecture was suggested, with four main parts: Shallow Feature Extraction (SFE), CNN/Transformer, Deep Feature Fusion (DFF), and up-sampling. This combination uses CNN to find local features and Transformers to find global dependencies, which makes deep feature learning strong and adaptable. The proposed model had an overall accuracy of 97.13%, a precision of 95.30%, a specificity of 96.74%, and an AUC of 98.74%. This shows that it is quite good at finding cancer. These results show that the model could provide accurate, quick, and easyto-use diagnostic solutions.

TransCat: a hybrid CNN-transformer network with KAN for medical image segmentation

Medical image segmentation is a fundamental task in computer vision and plays an important role in clinical diagnosis and treatment planning. Existing methods are mainly built on either convolutional neural networks (CNNs) or Transformer-based encoder-decoder architectures. CNNs are effective at capturing local patterns, whereas Transformers are better suited to modeling long-range dependencies. However, current hybrid designs still face three persistent challenges: insufficient exploitation of multi-scale CNN features for global modeling, rapidly increasing attention cost after token expansion, and ineffective fusion between CNN and Transformer representations. To address these issues, we propose TransCat, a hybrid CNN-Transformer architecture for medical image segmentation. First, we introduce Patch Concat, which converts multi-scale CNN features into patch tokens and feeds them jointly into the Transformer, enabling global reasoning across multiple scales. Second, to control the computational burden caused by the enlarged token set, we develop an extended deformable attention mechanism with attentive value identification. Third, we design a Kolmogorov–Arnold Network (KAN) attention module for adaptive cross-branch fusion. The module projects high-level CNN and Transformer features into a shared space and uses a KAN layer to estimate data-dependent pixel-wise weights. These weights selectively modulate the globally modeled Transformer features, allowing complementary local structural evidence from the CNN branch to be incorporated according to the image content rather than through fixed addition or concatenation. Experiments on multiple medical image segmentation benchmarks show that TransCat is competitive under the reported protocols, with its clearest advantage observed on unseen polyp test data.

Comparative Study of CNN, Hybrid, and Transformer Architectures in Medical Image Classification.