MS2FTNet: Multiscale Spatial–Spectral Fusion Transformer Network for Hyperspectral Image Classification
Abstract
Hyperspectral image classification (HSIC) aims to discriminate land cover categories accurately by exploiting rich spatial and spectral information. However, existing methods still suffer from insufficient collaborative modeling of spatial and spectral features, as well as inadequate fusion of local and global information, leading to limited feature extraction and restricted spatial-spectral correlation modeling. To address these issues, a Multiscale Spatial-Spectral Fusion Transformer Network (MS2FTNet) for HSIC is proposed in this article. First, a dual-branch multiscale feature extraction structure is constructed, in which the spatial branch integrates Fourier convolution with learnable Gaussian weights to model multiscale spatial context and enhance key region responses, while the spectral branch employs multiscale 3D depthwise convolutions and spectral weight calibration to extract discriminative spectral features. Second, a Convolution Pooling Transformer module is designed to preserve local spatial-spectral details while capturing long-range cross-band dependencies. Third, a Spatial Center Token Transformer module is proposed to model relationships among multiscale neighborhoods centered on the target pixel and adaptively aggregate spatial context relevant to its classification, thereby enhancing its feature representation. Experimental results on four public datasets demonstrate that the proposed MS2FTNet achieves the best classification accuracy across different scenes, significantly outperforming several state-of-the-art methods and exhibits strong robustness and generalization capability, particularly under small training sample conditions. The code of MS2FTNet is available at https://github.com/liangjiaqi-4008/MS²FTNet