Skip to content
Open access

A Hybrid Vision Transformer and U-Net Framework for Automated Cardiac MRI Segmentation and Abnormality Detection

2026 · Journal of Machine Learning Innovations and Artificial Intelligence Horizons · Vol 01, pp. 01-20 · 0 citations

TL;DR

A hybrid framework that integrates a Vision Transformer encoder with a U-Net decoder for automated cardiac MRI segmentation and abnormality detection and demonstrates significant potential for improving automated cardiac assessment in clinical workflows.

Abstract

Cardiovascular diseases remain a leading cause of mortality worldwide, and accurate segmentation of cardiac structures from MRI is critical for clinical diagnosis. We propose a hybrid framework that integrates a Vision Transformer encoder with a U-Net decoder for automated cardiac MRI segmentation and abnormality detection. The encoder divides input images into non-overlapping patches and processes them through multi-head self-attention layers, thereby capturing long-range dependencies and global anatomical context. These encoded features are then passed to the U-Net decoder, where skip connections preserve high-resolution spatial details from corresponding encoder levels. This design enables the decoder to combine global semantic information with local structural cues for precise pixel-wise segmentation of the left ventricle, right ventricle, and myocardium. A composite loss function, balancing Dice loss and cross-entropy loss, is employed to handle class imbalance and ensure robust pixel discrimination. Following segmentation, quantitative biomarkers such as ventricular volumes, ejection fraction, and wall thickness are extracted from the predicted masks. These features are subsequently fed into a fully connected neural network for binary classification of examinations as normal or abnormal. Furthermore, attention maps derived from the transformer layers provide interpretability by highlighting image regions that most influence the diagnostic decision. The proposed method is evaluated on short-axis cine MRI data with rigorous patient-level data partitioning and extensive augmentation. Our contributions include a novel hybrid architecture that effectively merges global and local feature representations, a clinically relevant abnormality detection pipeline, and an interpretability mechanism that aligns with expert annotations. This work demonstrates significant potential for improving automated cardiac assessment in clinical workflows.

Read PDF

Similar papers

Open access Sep 2026

Global Self-Attention for Cardiac MRI: Unified Segmentation and Interpretable Pathology Diagnosis

A fully supervised framework that couples a Vision Transformer encoder with a convolutional decoder to segment the left ventricle (LV), right ventricle (RV), and myocardium (Myo) on the Automated Cardiac Diagnosis Challenge (ACDC) dataset indicates that transformer-based encoders provide a competitive and interpretable...

Faizan Ahmad, Muhammad Anwar · 0 citations
Open access Oct 2026

Boundary Aware Multi Scale Attention U Net for Automated Cardiac MRI Segmentation

Accurate segmentation of cardiac structures in cine cardiac magnetic resonance (CMR) images is essential for quantitative assessment of cardiac function and disease. However, automatic delineation of the right ventricle, myocardium, and left ventricle remains challenging because of anatomical scale variations, weak tis...

S. Bharadwaj, Wilson Singh · 0 citations
Sep 2026

Lightweight Vision Transformer-Based U-Net for Brain Tumor Segmentation from MRI

A lightweight Vision Transformer UNet is proposed that combines the hierarchical feature extraction capability of UNet with the global context modeling of Vision Transformers, enabling effective learning of both local and global features while maintaining computational efficiency with only 2.6 million trainable paramet...

Sheekar Banerjee, Monika Chowdhury, M. Akash et al. · 0 citations
Open access Sep 2026

Development of Hybrid Convolutional Transformer Networks for Automated Segmentation of Volumetric Medical Imagery

The automatic segmentation of volumetric medical images is an essential task in computer-aided diagnosis, clinical decision support, treatment planning and quantitative biomedical analysis. Three-dimensional medical structure segmentation, however, is still difficult due to low contrast, intensity inhomogeneity, compli...

R. Veerapandiyan, S. Thaiyalnayaki · 0 citations
Oct 2026

Deep Learning-Based Multi-Scale Convolutional Pyramid Network for Internal Cardiac Structures Segmentation in Cine-MRI Volumes

Overall, MCP-Net provides a competitive and computationally efficient framework for automated multi-structure cardiac cine-MRI segmentation and external validation on the M&Ms benchmark without retraining or fine-tuning demonstrated the effectiveness and cross-dataset generalization capability of MCP-Net.

A. Kermi, Imane Cherifi, M. T. Khadir · 0 citations
Open access Oct 2026

Asymmetric encoder-decoder U-net with hybrid attention and residual dilated convolution for brain tumor segmentation

A novel HCSA-RMSDC U-Net is designed in this work, which is characterized by an asymmetric architecture with a computationally heavy encoder and a lightweight decoder that supports the complementary roles of the HCSA and RMSDC mechanisms in enhancing feature representation and multi-scale contextual modeling.

Xiao-Shuai Cao, Xi-Ge Song, Jin-Can Zhang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.