Learnable Frequency-Domain Attention with Deep Supervision for Accurate Skin Lesion Segmentation and Diagnosis

Abstract

In computer-aided diagnosis, accurate skin lesion segmentation is a key step; however, current transformer-based models often fail with blurred or fragmented boundaries, especially in challenging skin lesion images. In this work, we propose a Swin Transformer framework enhanced with a novel Frequency-Aware Multi-Scale Cross-Scale Attention module, named FreqMSCSAM-SwinSegNet. This module separates features into structural (low-frequency) and edge (high-frequency) information, therefore applying an independent attention mechanism to each form of feature information. It also includes a learnable edge extractor, cross-attention skip connections, a dual-attention decoder, and a hybrid loss function that combines Dice, Focal, and boundary losses, and an auxiliary morphological consistency loss that enforces clinically plausible lesion shapes. We evaluate the model on ISIC2018, where it reaches an IoU of 0.9228, a Dice coefficient of 0.9406, and an Accuracy of 0.9713, beating DeepLabV3++, SegFormer, and UNet++. The model also performs well on the ISIC2017 dataset, showing strong generalization. Overall, FreqMSCSAM-SwinSegNet offers a robust and practical solution for clinical skin lesion segmentation, with the potential to improve diagnostic accuracy and early melanoma detection.