Hybrid CNN–BiLSTM with Multi-Transformer Stacking for Skin Lesion Classification
TL;DR
A dual-branch deep learning framework that integrates a CNN–BiLSTM module for local spatial–sequential feature modeling with multiple transformer models (ViT, DeiT, SwinV2, SwinV2, and BEiT) for global representation learning is proposed.
Abstract
Skin cancer is one of the most common diseases worldwide and, if left untreated, it can be life threatening. In this work, we propose a dual-branch deep learning framework that integrates a CNN–BiLSTM module for local spatial–sequential feature modeling with multiple transformer models (ViT, DeiT, SwinV2, and BEiT) for global representation learning. Each input image is processed in parallel by two branches. The first branch consists of a lightweight convolutional network, followed by row-wise and column-wise Bidirectional LSTM layers to capture spatial and sequential dependencies. The second branch employs a high-resolution transformer to extract features from the image. The features extracted from both branches of the hybrid CNN–BiLSTM–Transformer model are fused to construct a unified classifier for accurate skin lesion recognition. To further improve robustness, four CNN–BiLSTM–Transformer hybrid models are combined using a stacking strategy based on logistic regression that aggregates their prediction probabilities. Experimental results on the binary Kaggle and multi-class HAM10000 datasets demonstrate that stacking achieves better performance than the individual hybrid CNN–BiLSTM–Transformer models, reaching accuracies of 91.21% on the binary Kaggle dataset and 86.08 ± 0.47% on the HAM10000 dataset, respectively. These findings confirm the efficiency of using complementary local and global features for skin lesion classification.