FuseMD-XNet: Uncertainty-aware multi-modality fusion network with multilevel visual explanations for skin cancer diagnosis.

Abstract

Skin cancer is a prevalent and potentially fatal disease that emphasizes the need for accurate and interpretable diagnostic tools to improve patient outcomes. Although DL has advanced automated skin lesion analysis, most models rely solely on dermoscopic images and neglect the complementary clinical metadata. In this study, we propose FuseMD-XNet, a multimodal transformer-based framework that integrates dermoscopic images with structured patient metadata. The model employs intermediate fusion through feature concatenation and cosine similarity alignment, followed by adaptive certainty-guided fusion to dynamically weigh the modality contributions based on confidence estimates. To ensure transparency, the FuseMD-XNet incorporates multilevel explainability using ShapleyCAM, FinerCAM, and SHAP methods. The efficacy of FuseMD-XNet was validated on the PAD-UFES-20 dataset, where it achieved an overall mean diagnostic accuracy of 94.4±0.8% and a mean AUC of 95.9±0.5% across all lesion classes. The highest class-specific performance was observed for basal cell carcinoma (BCC), with an accuracy of 98.4±0.4% and an AUC of 98.7±0.3%, whereas melanoma achieved an accuracy of 97.9±1.8% and an AUC of 98.2±0.5%. On the ISIC 2019 dataset, FuseMD-XNet demonstrated strong generalization performance with an overall mean accuracy of 93.0±0.8% and a mean AUC of 94.6±0.6%, whereas melanoma achieved a class-specific accuracy of 94.7±1.6% and an AUC of 96.3±1.1%. Additionally, an integrated risk stratification module enabled personalized assessments validated by feature importance analysis. These results demonstrate the potential of FuseMD-XNet to improve the classification accuracy and interpretability of skin cancer.