Multiresolution Infrared and Visible Image Fusion via Implicit Neural Representations

Abstract

Dual-spectral measurement instruments integrate complementary thermal radiation and textural information, thereby alleviating the information acquisition limitations of single-modality sensors. Due to detector manufacturing constraints and hardware cost, the infrared sensors usually provide lower spatial sampling rates than visible sensors. The existing methods commonly rely on interpolation-based upsampling to align the resolutions of infrared and visible images. However, this preprocessing step may introduce smoothing effects and local structural degradation, which can affect subsequent fusion results and downstream measurement tasks. To address this practical imaging problem, this article proposes a two-stage multiresolution infrared and visible image fusion framework based on implicit neural representations (INRs). In Stage I, the modality-specific implicit representation modules are used to learn continuous representations of infrared and visible images, and resolution-consistent bimodal representations are generated on a common target coordinate grid. In Stage II, the INRsFuse module integrates infrared thermal intensity information with visible structural and chrominance information in a shared normalized coordinate space to generate the final fused image. Experiments are conducted on public infrared-visible image fusion datasets, including M3FD, multi-spectral road scenarios (MSRS), and Roadscene. In addition, downstream semantic segmentation experiments on the MSRS dataset are used to evaluate the support of fused results for pixel-level target localization. Experimental results show that the proposed method achieves competitive fusion quality and obtains 93.08 % mean accuracy (mAcc) and 86.55 % mean intersection over union (mIoU) in the downstream semantic segmentation evaluation.