DenseSwinLight: A Hybrid CNN–Transformer Model with Lightweight Post-hoc Fusion for Visual Explainability

Abstract

The Hybrid CNN–Transformer architectures offer high-performance image analysis by combining local extraction and global contextual modeling. However, this integration complicates decision-making mechanisms and increases the opacity of models, making them difficult for human users to interpret. We propose DenseSwinLight, a hybrid approach that combines convolution-based feature extraction using DenseNet201 and global contextual modeling using the Swin Transformer V2-Large to simultaneously enhance local discrimination and global context. Beyond performance, the goal of this work is to make the model's decisions more transparent. We introduce a post-hoc explainability module based on the fusion of two complementary sources of evidence, namely a Grad-CAM map from the CNN branch and a proxy attention map. The fusion is learned by an extremely lightweight MLP fusion module, consisting of only 65 parameters, and constrained by area, total variation, and entropy regularizations to produce a parsimonious and stable explanatory mask. Evaluated on a real dataset acquired in an uncontrolled environment covering six classes (Bacteria, Fungi, Healthy, Pest, Phytophthora, and Virus), DenseSwinLight achieves an accuracy of 98.96% on the test set with Test-Time Augmentation, and a macro F1 score of 0.9897. The fusion module achieves a combined soft fidelity score of 0.498 and 0.513 in hard mode according to the insertion and deletion curves, for an average mask area of 0.12. These results confirm the model's ability to provide robust predictions while producing compact and actionable visual explanations for human interpretation.