Skip to content

The PID-Aware Geometric Semantic Representation Learning for Infrared-Visible Image Fusion

2026 · IEEE Transactions on Geoscience and Remote Sensing · Vol 64, pp. 4414714-4414714 · 0 citations · 51 references

Abstract

To relieve the problems of the lack of semantic alignment in the geometric representation space and insufficient modality-specific instruction calibration in infrared-visible image fusion (IVIF), we first propose a dynamic instruction-aware geometric representation (DIGR) for IVIF, which achieves semantic alignment between fusion features and semantic features within the geometric space and progressively injects textual instructions into the deep-layer features. Second, to alleviate the semantic gap between fusion features and semantic features, we propose a cross-graph neighborhood interaction (CGNI) module. In this module, each node in one graph is updated by aggregating information from the neighbor nodes of its position-aligned counterpart in the other graph, thereby enabling cross-level semantic alignment. Third, to alleviate the absence of modality-related text correction in the fusion network, we introduce an instruction-guided proportional–integral–derivative (IPID) calibration module, which uses modality-aware texts as external priors. By modeling the discrepancy between instructions and deep-level modality features using the proportional–integral–derivative (PID) control principle, IPID achieves progressive and controllable instruction calibration. Extensive experiments on three public datasets, namely, MFNet, Potsdam, and WHU, demonstrate that the proposed DIGR achieves superior fusion results and segmentation performance. The code of DIGR is publicly available at https://github.com/YongzheWang666/DIGR

View source

Similar papers

Preprint Oct 2026

DTFormer: Text-Guided Semantic Alignment for RGB-D Segmentation

RGB-D semantic segmentation has made notable progress by fusing RGB and Depth, yet mainstream models still learn features almost exclusively from pixel-level supervision, lacking direct high-level semantic constraints. This raises a central question-can external knowledge such as language priors inject stronger semanti...

Zi-Chuan Wei, Yin-Long Liu, Yan Xia et al. · 0 citations
Open access Aug 2026

MISSFusion: a multi-granularity interaction and semantic-structure rectification network for infrared and visible image fusion

MISSFusion focuses on stage-aware integration of mid-level multi-granularity complementary interaction and higher-stage semantic-structure rectification, and introduces a Multi-Granularity Complementary Interaction Block (MCIB) to balance local details and regional context while exploiting consistency and discrepancy c...

Yuan-Yuan He, Xue-Yan Gao, Ruixi Liu et al. · 1 citation
Open access Aug 2026

AMCA-SGG: Adaptive multi-prototype learning and cross-modal alignment for unbiased scene graph generation

This work proposes AMCA, an unbiased SGG framework integrating adaptive multi-prototype learning with cross-modal alignment, which achieves consistently competitive performance across multiple SGG tasks, with particularly strong improvements on the unbiased mR@K metric.

Jinhao Fan, Yuan-Hao Xi, Chuanping Hu et al. · 0 citations
Open access Sep 2026

Boundary-Guided Dual-Perspective Cross-Modal Fusion Network for RGB-IR Object Detection

A Boundary-Guided Dual-Perspective Cross-Modal Fusion Network (BDPNet) is proposed to explicitly preserve shallow geometric structures and decouple deep semantic fusion into macroscopic and microscopic perspectives.

Hu Lin, Zhi-Wei Fu, Xiu-Mei Chen et al. · 0 citations
Conference Open access Sep 2026

Auxiliary text-guided image restoration for image-text matching

A new ITM framework that improves the model's discriminative performance by focusing on localized core attributes and has a better robustness in handling highly similar hard negatives, which provides a new way to address cross-modal hard sample discrimination.

Ya-Ting Liu, Kuang-Rong Hao · 0 citations
Open access Sep 2026

Vision–language guided semantic-geometric transformer for memory-efficient 3D scene understanding

This work proposes a segmentation framework guided by Contrastive Language–Image Pre-training (CLIP) that enriches sparse 3D tokens with vision–language semantic priors and introduces a decoupled CLIP-induced semantic residual that forms semantic-geometric attention biases for local window attention.

Li-Cheng Liu, Yu Li, Fu-Yong Liu · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.