Interpretable Dictionary Transfer-Based Spectral Super-Resolution for Similar-Scene Multispectral Images
Abstract
Deep learning-based transfer learning has been widely used in spectral super-resolution, which can transfer the general spectral mapping relationship learned from a pair of multispectral (MS) and hyperspectral (HS) images to other similar-scene MS images, to realize spectral super-resolution from one single MS image to the desired HS image. However, deep learning-based transfer learning requires large amounts of data for fine-tuning and suffers from poor interpretability, which may considerably reduce the effectiveness of spectral super-resolution. To alleviate this limitation, this article proposes an interpretable dictionary transfer-based spectral super-resolution method with closed-form solutions for similar-scene multispectral images, tries to adjust the prelearned spectral dictionary to fit with different task domain and to reconstruct a more accurate desired HS image. Specifically, first, taking typical spectral set in the source domain and similar spectral set in the task domain as a connection, a transfer matrix optimization strategy is proposed for the prelearned spectral dictionary. Second, an orthogonal matrix-based dictionary transfer scheme is established, to transfer each column of the spectral dictionary prelearned from the MS and HS image pairs, into the task domain of the observed similar-scene MS image for spectral super-resolution. Finally, the coefficient matrix of the transferred spectral dictionary is optimized as closed-form solutions, by using spatial information of the observed similar-scene MS image and the spectral similarity of the extracted spectral sets under sparse and low-rank constraints. Experimental results on both simulated and real datasets indicate that, our proposed method achieves superior performance and generalization ability across different scenarios, compared with the other state-of-the-art spectral super-resolution methods based on both deep learning and dictionary learning.