An Explainable Hybrid Deep Learning Framework Combining Fine-Tuned Convolutional Neural Network Ensembles for Dermoscopic Images of Skin Lesions

Abstract

Introduction: Accurate diagnosis in dermoscopic examination is challenging due to the visual similarity between skin lesion types and the variability within the same group. Early diagnosis and treatment of skin diseases, especially skin cancer, are vital. Objectives:  This study aimed to develop and evaluate an explainable multi-architecture framework integrating convolutional, transformer-based, hybrid, and ensemble learning approaches for multi-class skin lesion classification. Methods: Using HAM10000, four architectures (EfficientNet-B0, MobileNetV3-Large, ResNet34, DeiT-Small) were fine-tuned in two phases and combined into a validation-optimized, macro-F1-driven soft-voting ensemble; an independent ResNet34+XGBoost hybrid was evaluated in parallel. Lesion-grouped partitioning was used to prevent overlap between data subsets, with an additional image-level experiment assessing the influence of partitioning strategy on performance. Evaluation used accuracy, macro-averaged precision, recall, F1-score, per-class metrics, nested 5-fold cross-validation, McNemar's test, and lesion-clustered bootstrap effect sizes (95% CIs). Grad-CAM, Grad-CAM++, and LIME provided interpretability, with Grad-CAM localization evaluated against segmentation masks via Intersection over Union (IoU). Results: Under lesion-grouped evaluation, the ensemble achieved the highest performance, with 85.76% accuracy, 75.24% macro-F1, and 0.971 macro-AUC. Paired accuracy gains over comparators ranged from 2.36 to 7.57 percentage points, with all lesion-clustered 95% CIs excluding zero. ResNet34 yielded the highest mean Grad-CAM IoU (0.456). Conclusion: The ensemble achieved improved classification performance under lesion-grouped evaluation. Quantitative localization analysis showed measurable spatial overlap with lesion regions, although clinical usefulness requires reader studies and external validation.