Attention-Enhanced Bimodal 3D Medical Image Segmentation with Two-Stage Learning
TL;DR
This work proposes an enhanced 3D segmentation framework, UAtten-Unetr, designed to improve segmentation accuracy and robustness in complex medical scenarios, and innovatively developed a unified loss function based on bimodal modality-specific Dice constraints and uncertainty regularization, optimized for synchronous learning across the ACDC and AMOS22 datasets.
Abstract
Computer-aided diagnostic technologies have demonstrated substantial advantages in 3D medical image segmentation, particularly in multimodal 3D medical image segmentation tasks, where they play a pivotal role in driving continuous innovation in related architectures. As an integration of U-Net and Transformer, the UNETR architecture has demonstrated remarkable efficacy in 3D medical image segmentation. Nevertheless, despite its successes, UNETR remains challenged by clinical complexities such as intricate tumor localization and anatomical structural diversity in complex clinical settings. To address these issues, we propose an enhanced 3D segmentation framework, UAtten-Unetr, designed to improve segmentation accuracy and robustness in complex medical scenarios. The framework captures global contextual information via hierarchical Transformer layers and incorporates a spatial–channel attention module to enable adaptive fusion of multimodal features, thereby effectively enhancing cross-modal feature alignment capabilities. Concurrently, we innovatively developed a unified loss function based on bimodal modality-specific Dice constraints and uncertainty regularization, optimized for synchronous learning across the ACDC (cardiac MRI) and AMOS22 (abdominal CT/MRI) datasets. Experimental results showed that UAtten-Unetr achieved an average Dice score of 92.20% on the ACDC dataset, exceeding the reported nnU-Net result of 91.61% by 0.59 percentage points. On the AMOS22 dataset, the proposed method achieved an average Dice score of 84.51%, exceeding the reported UNETR result of 78.33% by 6.18 percentage points. However, its myocardium Dice score (84.11%) was lower than those of nnU-Net (89.24%) and MT-UNet (89.04%), indicating a remaining limitation in myocardium boundary segmentation. These results indicate competitive segmentation performance under the reported experimental settings. This method delivers dual improvements in accuracy and generalization across complex anatomical scenarios, providing an effective solution for precise diagnosis in intricate clinical environments.