Skip to content
Book Open access

Learning Robust 3D Gaussian Representations via Discrepancy-Aware Infrared and Visible Modality Fusion

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · pp. 11498-11507 · 0 citations

Abstract

Infrared and visible image fusion (IVIF) combines robust infrared sensing with the rich structural details of visible imagery. Existing IVIF methods are predominantly developed in 2D image space, limiting them to fixed viewpoints and hindering explicit modeling of modality discrepancies at the scene level. Although recent works introduce multimodal cues into 3D representations, they focus primarily on cross-modality rendering conflicts or modality-dominant reconstruction rather than fusion-oriented selective preservation and suppression. We therefore propose Discrepancy-Aware Fusion with 3D Gaussian Splatting (DAF-GS), which learns robust 3D Gaussian representations through discrepancy-aware infrared and visible modality fusion. Specifically, a discrepancy-aware modality-guided cue mechanism estimates modality dominance from current-view responses and long-term modality dependency statistics, and incorporates the resulting cue into pixel-wise color rendering for explicit discrepancy-aware optimization. A primitive-wise detail compensation module further restores complementary fine-grained details weakened during selective suppression. Extensive experiments demonstrate the robustness of the learned representations and clear advantages over 2D fusion methods for novel-view synthesis across multiple datasets. The code is available at https://github.com/RollingPlain/DAF-GS.

Read PDF

Similar papers

Open access Sep 2026

Boundary-Guided Dual-Perspective Cross-Modal Fusion Network for RGB-IR Object Detection

A Boundary-Guided Dual-Perspective Cross-Modal Fusion Network (BDPNet) is proposed to explicitly preserve shallow geometric structures and decouple deep semantic fusion into macroscopic and microscopic perspectives.

Hu Lin, Zhi-Wei Fu, Xiu-Mei Chen et al. · 0 citations
Preprint Oct 2026

PCDFusion: Proposal-Context-Detail Bayesian Rendering for Infrared-Visible Image Fusion

Infrared-visible image fusion (IVIF) aims to combine visible and thermal infrared observations for human perception and downstream vision tasks. Visible and thermal infrared observations are physically complementary but unreliable in different ways. Existing deep IVIF methods mainly improve feature interaction and reco...

Rui Liu, Lin Gu, He-Song Li et al. · 0 citations
Book Open access Nov 2026

Infrared-Visible Image Fusion with Dual Degradation-Aware Priors

Degraded infrared-visible image fusion (DIVF) aims to integrate multimodal images under degradation factors. Existing methods typically address a single degradation type, limiting generalizability. Recent approaches have adopted large language models (LLMs) to simulate user instructions, yielding linguistic priors to h...

Xinyi Zeng, Wen-Xia Yin, Jia-Qi Cui et al. · 0 citations
Book Open access Nov 2026

Adaptive Infrared-Visible Image Fusion via Uncertainty-Aware Prompt Routing and Task-Tailored Bidirectional Vision Mamba

Infrared--visible image fusion seeks to retain structures shared by the two modalities while preserving their complementary cues. The appropriate fusion rule, however, is not uniform across spatial locations or feature scales. Regions in which the modalities agree mainly require structure preservation, whereas regions...

Ze-Yu Chen, Fei Gao, Li-Bo Weng · 0 citations
Book Open access Oct 2026

Multi-View Structural Representations for Physically-Consistent TeX Generation

Infrared and visible images provide complementary cues for low-light image generation, and physics-based methods can better exploit this complementarity. However, existing physics-based methods still rely heavily on paired data. To address these limitations, we propose Multi-View Routing Mixture-of-Experts (MVR-MoE), a...

Dao-Heng Li, Jian-Jun Qian, Meng-Kai Yan et al. · 1 citation
Sep 2026

RealFusion: Rethinking Infrared and Visible Image Fusion From a Modality-Invariant Perspective

Existing fusion methods integrate complementary information from source images to generate a mixed-modality fused image, causing visually ambiguous representations that degrade downstream task performance (e.g., segmentation and detection). Therefore, we rethink infrared and visible image fusion from a modality-invaria...

He Cui, Wen-Da Zhao, Hai-Peng Wang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.