HybridSkinLes: An Explainable CNN-and Transformer-Based Multi-Class Skin Lesion Classification Framework
TL;DR
Overall results indicate that hybrid CNN–Transformer architectures present a viable path for automated skin lesion categorization and Grad-CAM++ is integrated to enhance model understanding and promote medical confidence.
Abstract
Skin cancer is considered a deadly disease globally, and the timely identification of the disease may save human life. This research presents a CNN–Transformer-based fusion framework for automated multi-class skin lesion classification. This approach combines ResNet50 and Vision Transformer (ViT) to categorize skin lesions using the HAM10000 dataset. To assess their efficacy, a comparison with CNN and ViT models is also carried out. Seven classes of skin cancer are used for training the models, and class weighting is used to correct dataset asymmetry. According to the experimental dataset, the suggested hybrid framework shows improved performance over CNN and ViT, considering the accuracy (0.97) and macro-averaged F1-score (0.95). Furthermore, the efficiency of the suggested model is demonstrated by the fact that it delivers performance that is competitive with several existing approaches. Overall results indicate that hybrid CNN–Transformer architectures present a viable path for automated skin lesion categorization. Grad-CAM++ is integrated to enhance model understanding and promote medical confidence by enabling physicians to view visualizations that show the areas impacting the model’s conclusions. But there are still issues, including poor generalization, computational complexity, and a lack of external validation. Future research will concentrate on enhancing interpretability for practical implementation, integrating clinical information, and evaluating several datasets.