Dual-Branch Multi-Perspective Modulation Network for Efficient Infrared Image Super-Resolution
TL;DR
This work designs a multi-scale feature modulation enhancement unit (MFMEU) to capture cross-scale spatial features and a frequency-domain cross-correlation patch modulation unit (FCPMU) to explore global feature representations and develops an efficient bidirectional cross modulation unit (BCMU) to promote feature interaction between outputs of MFMEU and FCPMU.
Abstract
Transformer-based super-resolution methods achieve competitive performance by capturing non-local information through self-attention. Nevertheless, the computation of the self-attention introduces heavy computational overhead, and its inherent low-pass characteristic restricts the learning of local details. To address these problems, we propose an effective dual-branch multi-perspective modulation network (DMMN) for efficient infrared image super-resolution. Specifically, we design a multi-scale feature modulation enhancement unit (MFMEU) to capture cross-scale spatial features and a frequency-domain cross-correlation patch modulation unit (FCPMU) to explore global feature representations. We further develop an efficient bidirectional cross modulation unit (BCMU) to promote feature interaction between outputs of MFMEU and FCPMU. Extensive experimental results verify that DMMN achieves a competitive trade-off between reconstruction accuracy and computational efficiency. For instance, compared with the ×4 SRFormer-light, the proposed DMMN obtains an average gain of 0.08 dB in PSNR across five public test datasets, runs 2.7× faster, and uses only about 24% of the FLOPs.