ResUNet++ -ViT: A Fusion Framework for Accurate High-Resolution Medical Image Segmentation

Abstract

Segmentation of medical images is a crucial process for diagnosis and treatment planning. Yet, traditional CNN based models are often not able to convey high-level spatial relationships and intricate tissue boundaries in high-resolution medical images. To tackle these issues, this paper presents a novel ResUNet++–Vision Transformer (ResUNet++–ViT) framework incorporating both multi-scale local feature extraction and global contextual learning. The ResUNet++ backbone consists of residual blocks and nested skip connections, which are used to extract hierarchical features, and the Vision Transformer uses self-attention mechanisms to capture long-range dependencies. A fusion module allows for the integration of local and global features, resulting in better segmentation accuracy and preserving the boundary. The proposed model was tested on the ISBI 2012 Electron Microscopy Segmentation Challenge (EMSC) dataset, and obtained a Dice score of 0.960, IoU of 0.920, precision of 0.967, recall of 0.958, accuracy of 0.984 and Hausdorff distance of 2.76. The proposed framework is compared with FCN, UNet, Attention UNet, ResUNet, UNet++, Vision Transformer and TransUNet, and the results show its superiority. The results show that the combination of ResUNet++ and Vision Transformers greatly enhances the performance of segmentation, boundary delineation, and generalization in the field of advanced medical image analysis applications.