Skip to content
Book Open access

Adaptive Infrared-Visible Image Fusion via Uncertainty-Aware Prompt Routing and Task-Tailored Bidirectional Vision Mamba

Nov 2026 · 0 citations · 39 references

Abstract

Infrared--visible image fusion seeks to retain structures shared by the two modalities while preserving their complementary cues. The appropriate fusion rule, however, is not uniform across spatial locations or feature scales. Regions in which the modalities agree mainly require structure preservation, whereas regions with large cross-modal differences benefit from stronger complementary enhancement. We therefore introduce a coarse-to-fine framework with two coupled modules: an Uncertainty-Aware Dynamic Prompt Router (UDPR) and a Task-Tailored Bidirectional Vision Mamba (TT-BVM) block. UDPR predicts hierarchical uncertainty maps and uses them to route prompts across scales. TT-BVM uses the routed prompt and uncertainty map to separate common and discrepancy features, then models the two branches with dedicated bidirectional state-space blocks. The model is trained with an uncertainty-conditioned reconstruction loss and auxiliary terms that regularize routing and branch coordination. When trained only on MSRS, it obtains the best score in 16 of 20 metric entries across four public infrared--visible fusion benchmarks. The same model also performs competitively on three medical fusion benchmarks without retraining and improves semantic segmentation and object detection on fused images. These results support the uncertainty-guided design, and the mechanism analysis shows stable routing and branch behavior in the evaluated setting.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.