Swin-DeepLabV3: Enhanced Semantic Segmentation Through Global-Local Feature Fusion Using Swin Transformer and Atrous Spatial Pyramid Pooling
TL;DR
Swin-DeepLabV3 is proposed, a hybrid semantic segmentation framework that integrates global and local feature modeling by combining a hierarchical Swin Transformer encoder with an Atrous Spatial Pyramid Pooling-based context module, demonstrating an effective balance between contextual representation and spatial precision for medical image segmentation tasks.
Abstract
Accurate medical image segmentation relies on precise boundary delineation and effective contextual modeling across multiple spatial scales. Conventional convolutional neural network-based architectures, such as U-Net, are effective at capturing local spatial details but are limited in modeling long-range dependencies due to their restricted receptive fields. Transformer-based approaches have recently shown potential in addressing this limitation by enabling global feature interactions; however, their direct application to dense prediction tasks often introduces high computational cost and challenges in multi-scale representation learning. This paper proposes Swin-DeepLabV3, a hybrid semantic segmentation framework that integrates global and local feature modeling by combining a hierarchical Swin Transformer encoder with an Atrous Spatial Pyramid Pooling-based context module. The Swin Transformer encoder employs shifted window self-attention to capture long-range dependencies while preserving hierarchical multi-scale representations. To enhance contextual understanding, the Atrous Spatial Pyramid Pooling module is incorporated at the bottleneck to aggregate spatial information at multiple dilation rates. A lightweight decoder with skip connections is then used to fuse high-level semantic features with low-level spatial details, supporting improved boundary localization. The proposed approach is evaluated on three public breast ultrasound datasets, namely BUSI, BUS-B, and BUS-BRA, under a 5-fold cross-validation protocol. Experimental results show that Swin-DeepLabV3 achieves strong overlap performance across datasets, with the best Dice scores of 83.54%, 88.49%, and 77.13% on BUS-B, BUS-BRA, and BUSI, respectively. The proposed architecture demonstrates an effective balance between contextual representation and spatial precision for medical image segmentation tasks. The implementation of the proposed method is publicly available at https://github.com/CaoMinhh/Swin-DeepLabV3