2026· IEEE Transactions on Geoscience and Remote Sensing· Vol 64, pp. 4414714-4414714· 0 citations· 51 references
Abstract
To relieve the problems of the lack of semantic alignment in the geometric representation space and insufficient modality-specific instruction calibration in infrared-visible image fusion (IVIF), we first propose a dynamic instruction-aware geometric representation (DIGR) for IVIF, which achieves semantic alignment between fusion features and semantic features within the geometric space and progressively injects textual instructions into the deep-layer features. Second, to alleviate the semantic gap between fusion features and semantic features, we propose a cross-graph neighborhood interaction (CGNI) module. In this module, each node in one graph is updated by aggregating information from the neighbor nodes of its position-aligned counterpart in the other graph, thereby enabling cross-level semantic alignment. Third, to alleviate the absence of modality-related text correction in the fusion network, we introduce an instruction-guided proportional–integral–derivative (IPID) calibration module, which uses modality-aware texts as external priors. By modeling the discrepancy between instructions and deep-level modality features using the proportional–integral–derivative (PID) control principle, IPID achieves progressive and controllable instruction calibration. Extensive experiments on three public datasets, namely, MFNet, Potsdam, and WHU, demonstrate that the proposed DIGR achieves superior fusion results and segmentation performance. The code of DIGR is publicly available at https://github.com/YongzheWang666/DIGR
RGB-D semantic segmentation has made notable progress by fusing RGB and Depth, yet mainstream models still learn features almost exclusively from pixel-level supervision, lacking direct high-level semantic constraints. This raises a central question-can external knowledge such as language priors inject stronger semanti...
Zi-Chuan Wei, Yin-Long Liu, Yan Xia et al.· 0 citations
MISSFusion focuses on stage-aware integration of mid-level multi-granularity complementary interaction and higher-stage semantic-structure rectification, and introduces a Multi-Granularity Complementary Interaction Block (MCIB) to balance local details and regional context while exploiting consistency and discrepancy c...
Yuan-Yuan He, Xue-Yan Gao, Ruixi Liu et al.· Journal of King Saud Univers...· 1 citation
This work proposes AMCA, an unbiased SGG framework integrating adaptive multi-prototype learning with cross-modal alignment, which achieves consistently competitive performance across multiple SGG tasks, with particularly strong improvements on the unbiased mR@K metric.
Jinhao Fan, Yuan-Hao Xi, Chuanping Hu et al.· Journal of King Saud Univers...· 0 citations
A Boundary-Guided Dual-Perspective Cross-Modal Fusion Network (BDPNet) is proposed to explicitly preserve shallow geometric structures and decouple deep semantic fusion into macroscopic and microscopic perspectives.
Hu Lin, Zhi-Wei Fu, Xiu-Mei Chen et al.· Remote Sensing· 0 citations
A new ITM framework that improves the model's discriminative performance by focusing on localized core attributes and has a better robustness in handling highly similar hard negatives, which provides a new way to address cross-modal hard sample discrimination.
Ya-Ting Liu, Kuang-Rong Hao· International Conference on...· 0 citations
This work proposes a segmentation framework guided by Contrastive Language–Image Pre-training (CLIP) that enriches sparse 3D tokens with vision–language semantic priors and introduces a decoupled CLIP-induced semantic residual that forms semantic-geometric attention biases for local window attention.