Aug 2026· Mathematics· Vol 14, pp. 3018· 0 citations· 33 references
TL;DR
A Frequency–Spatial Rectification Module that jointly models spatial-domain structures and frequency-domain representations in a unified manner and a Difference-driven Cross-modal Interaction Module (DCIM), where inter-modal discrepancies explicitly guide information exchange among modalities are designed.
Abstract
Multi-modal image fusion integrates complementary information from different modalities to generate a unified representation that is informative for human perception and beneficial to downstream vision tasks. However, existing methods often inefficiently model global features and their cross-modal interaction is insufficient, resulting in the suboptimal preservation of structural details and modality-specific information. To address these issues, we propose SACMFuse, a structure-aware cross-modal interaction network for multi-modal image fusion. SACMFuse is built upon the linear complexity attention mechanism of LAMA, which enables efficient and effective global feature modeling. In this study, LAMA is extended into a CrossLAMA mechanism to facilitate deep-level information interaction among modalities. Within this framework, we propose a Frequency–Spatial Rectification Module (FSRM) that jointly models spatial-domain structures and frequency-domain representations in a unified manner. By perceiving and adaptively rectifying structural features across modalities, FSRM enhances the structural consistency and discriminability of the fused features. Furthermore, to strengthen cross-modal complementarity, we design a Difference-driven Cross-modal Interaction Module (DCIM), where inter-modal discrepancies explicitly guide information exchange among modalities. This mechanism encourages the network to focus on complementary structures while suppressing redundant responses. Extensive experiments on multiple datasets demonstrate that the performance of SACMFuse is competitive for the majority of metrics, providing a robust and advanced solution for image fusion tasks.
Infrared-visible image fusion seeks a single representation that preserves thermal saliency while retaining visible-spectrum structure. Existing networks often perform fusion with a single interaction mechanism, leaving the modality gap, long-range scene consistency, and fine-grained cross-modal complementarity insuffi...
Jin-Ting Zhou, Guang Zeng, Bo Hu et al.· 2026 23rd International Comp...· 0 citations
Multi-modality image fusion (MMIF) seeks to generate images that are suitable for human viewing while also supporting downstream visual understanding. Existing approaches usually learn fusion and segmentation together, either through cascaded task supervision or by exchanging intermediate features. In most cases, howev...
Cheng-Chao Wang, Xue Wang, Hao-Yu Xiong et al.· Proceedings of the 34th ACM...· 0 citations
A multi-modal cross-attention cross-attention fusion network named MCF-Net is proposed, based on cross-modal attention and difference convolution, designed for tri-modal fusion and remains effective for bi-modal settings.
Ming-Chun Li, Xin-Ran Wu, Rui Wang et al.· Electronics· 0 citations
Infrared and visible image fusion combines complementary thermal and structural information from the two modalities into a single composite image. Existing methods have two critical limitations: (1) inadequate utilization of visible structural information causes blurred edges, and (2) modality-specific and shared respo...
Shun-Li Liu, An-Jie Chen, Qiao Luo et al.· Italian National Conference...· 0 citations
Sign language recognition serves as essential interaction way between hearing-impaired and non-signing communities. However, sign language is inherently a multi-modal interactive process, requiring the complex interplay of spatial morphology and dynamic kinematics. While multi-modal approaches show promise, effectively...
Jin-Yang Feng, Zhong Guan, Yong-Li Hu et al.· Proceedings of the 28th Inte...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.