TRT-GLA: Tri-Representation Transformers with Global-Local Attention for High-Fidelity Multi-Modal MRI Super-Resolution

Abstract

The super-resolution (SR) of Magnetic Resonance Imaging (MRI) is essential for utilizing clinical scans with limited resolution, noise, and anisotropic sampling, such as multi-modal brain tumor imaging. In this work, we propose a Tri-Representation hybrid framework for MRI SR, TRT-GLA, that redefines the MRI SR task as a joint spatial–spectral–structural high-resolution image generation problem. TRT-GLA utilizes (i) spatial global–local attentions for modeling the spatial anatomy, (ii) a Fourier spectral transfer mechanism for upholding spectral consistency, and (iii) multi-scale hierarchical spectral decomposition for improved edge details. To adapt the learning framework to medical imaging characteristics, we introduce a tri-representation consistent loss function that explicitly combines pixel-wise, spectral, and edge structure priors from the high-resolution ground-truth, as well as a progressive resolution learning strategy. Our large-scale brain tumor experiments, on the IXI, BraTS 2019, 2020, and 2023 datasets, show that TRT-GLA achieves state-of-the-art results at upsampling factors of ×2, ×4, and ×8, respectively, achieving substantial improvements across CNN, GAN, and transformer-based methods in Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Multi-scale Structural Similarity Index (MS-SSIM). We further demonstrate how SR benefits brain tumor segmentation through the downstream task evaluation of a dual-branch segmentation framework. TRT-GLA produces highly accurate tumor segmentation results from low-resolution inputs, improving over native high-resolution inputs at ×8 in critical tumor boundary regions and in small tumor regions. There remains a small gap between native, high-resolution imaging and SR-enhanced performance, which TRT-GLA nearly closes under realistic scenarios. Our results highlight the importance of synthesizing unified priors over spatial, spectral, and structural domains within a transformer for anatomically faithful reconstructions. Importantly, we also establish the utility of TRT-GLA in supporting quantitative analysis through a downstream tumor segmentation experiment that is clinically relevant.