RWKVHSI: Dual-Stream Recurrent Spectral–Spatial Evidence Fusion for Efficient Hyperspectral Image Classification
Abstract
Hyperspectral image (HSI) classification requires reconciling highly correlated spectral responses with spatially heterogeneous structures while controlling computational cost. Convolutional networks provide strong local inductive biases but capture long-range context only indirectly, whereas self-attention improves global interaction at quadratic token cost. In this article, we introduce RWKVHSI, a compact dual-stream recurrent framework that models complementary pointwise channel-transformed and spatial-structural evidence. A shared stem first embeds each HSI patch. The pointwise channel-transformation stream operates independently at each spatial location, while the spatial-structural stream extracts local neighborhood patterns through lightweight convolution. Both representations are serialized row-wise and processed by independently parameterized recurrent weighted key–value blocks whose state updates are linear in sequence length and avoid pairwise attention maps. An adaptive channel-fusion module derives global branch descriptors, recalibrates their responses, and integrates them through residual learning. Experiments on Indian Pines, Houston2013, and WHU-Hi-LongKou show that RWKVHSI achieves competitive or superior accuracy relative to representative convolutional, transformer, and state-space baselines while maintaining a compact computational footprint. Ablation and sensitivity analyzes further verify the complementary contributions of the two streams, adaptive fusion, recurrent depth, and residual optimization. These results demonstrate an effective accuracy–efficiency tradeoff for within-scene HSI classification through recurrent spectral–spatial evidence decomposition.