Skip to content
Open access

SACMFuse: Structure-Aware Cross-Modal Interaction Network for Multi-Modal Image Fusion

Aug 2026 · Mathematics · Vol 14, pp. 3018 · 0 citations · 33 references

TL;DR

A Frequency–Spatial Rectification Module that jointly models spatial-domain structures and frequency-domain representations in a unified manner and a Difference-driven Cross-modal Interaction Module (DCIM), where inter-modal discrepancies explicitly guide information exchange among modalities are designed.

Abstract

Multi-modal image fusion integrates complementary information from different modalities to generate a unified representation that is informative for human perception and beneficial to downstream vision tasks. However, existing methods often inefficiently model global features and their cross-modal interaction is insufficient, resulting in the suboptimal preservation of structural details and modality-specific information. To address these issues, we propose SACMFuse, a structure-aware cross-modal interaction network for multi-modal image fusion. SACMFuse is built upon the linear complexity attention mechanism of LAMA, which enables efficient and effective global feature modeling. In this study, LAMA is extended into a CrossLAMA mechanism to facilitate deep-level information interaction among modalities. Within this framework, we propose a Frequency–Spatial Rectification Module (FSRM) that jointly models spatial-domain structures and frequency-domain representations in a unified manner. By perceiving and adaptively rectifying structural features across modalities, FSRM enhances the structural consistency and discriminability of the fused features. Furthermore, to strengthen cross-modal complementarity, we design a Difference-driven Cross-modal Interaction Module (DCIM), where inter-modal discrepancies explicitly guide information exchange among modalities. This mechanism encourages the network to focus on complementary structures while suppressing redundant responses. Extensive experiments on multiple datasets demonstrate that the performance of SACMFuse is competitive for the majority of metrics, providing a robust and advanced solution for image fusion tasks.

Read PDF

Similar papers

Conference Open access Sep 2026

MDFN: A Multi-Scale Dual-Attention Fusion Network For Infrared And Visible Images

Infrared-visible image fusion seeks a single representation that preserves thermal saliency while retaining visible-spectrum structure. Existing networks often perform fusion with a single interaction mechanism, leaving the modality gap, long-range scene consistency, and fine-grained cross-modal complementarity insuffi...

Jin-Ting Zhou, Guang Zeng, Bo Hu et al. · 0 citations
Book Open access Nov 2026

SDFMod: Semantics-guided Dynamic Frequency Modulation for Multi-modality Image Fusion

Multi-modality image fusion (MMIF) seeks to generate images that are suitable for human viewing while also supporting downstream visual understanding. Existing approaches usually learn fusion and segmentation together, either through cascaded task supervision or by exchanging intermediate features. In most cases, howev...

Cheng-Chao Wang, Xue Wang, Hao-Yu Xiong et al. · 0 citations
Open access Sep 2026

SFTINet: Spatial–Frequency Token Interaction for Multi-Modal Object Re-ID

By seamlessly integrating spatial–frequency interaction with token-level masking, SFTINet effectively suppresses background interference while preserving cross-modal complementary information.

Lin Qi, Tian-Cun Guo, Yan-Fei Dong et al. · 0 citations
#edge computing Open access Sep 2026

Edge-Aware Dynamic Convolution and Cross-Modal Relation-Aware Fusion Network for Infrared and Visible Image Fusion

Infrared and visible image fusion combines complementary thermal and structural information from the two modalities into a single composite image. Existing methods have two critical limitations: (1) inadequate utilization of visible structural information causes blurred edges, and (2) modality-specific and shared respo...

Shun-Li Liu, An-Jie Chen, Qiao Luo et al. · 0 citations
Book Open access Oct 2026

M3CA: Multi-modal and Multi-view Fusion based on Multi-layer Cross Attention for Isolated Sign Language Recognition

Sign language recognition serves as essential interaction way between hearing-impaired and non-signing communities. However, sign language is inherently a multi-modal interactive process, requiring the complex interplay of spatial morphology and dynamic kinematics. While multi-modal approaches show promise, effectively...

Jin-Yang Feng, Zhong Guan, Yong-Li Hu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.