Enhancing Transformer-Based RGB-T Image Fusion via Hybrid Pixel Upsampling

Abstract

Guided thermal super-resolution (GTSR) is a cross-modal deep learning task that reconstructs high-resolution thermal images from low-resolution inputs using structural guidance from high-resolution RGB images. This problem is relevant to intelligent systems and data-driven applications where highresolution thermal sensors are limited by hardware constraints. Recent transformer-based fusion models achieve strong cross-modal performance but rely on fixed bicubic interpolation during early upsampling, which can degrade high-frequency spatial details prior to feature extraction. In this work, a hybrid upsampling module is proposed that combines learnable PixelShufflebased feature expansion with a bicubic residual skip connection, integrated into a SwinFuSR-based pipeline without modifying the backbone architecture. Experiments on the CIDIS dataset at scale factor $\times 2$ show that the proposed method achieves comparable PSNR to Lanczos interpolation on the validation set (44.302 vs $44.305 d B)$, with a statistically significant but marginal advantage on the held-out test set $(p=0.029$, paired t-test, $n=100)$, while maintaining low computational overhead (+1.5% GFLOPs). An ablation study confirms that the bicubic residual skip is the dominant contributor, with the learnable PixelShuffle module providing a consistent additional improvement.