Adaptive Infrared-Visible Image Fusion via Uncertainty-Aware Prompt Routing and Task-Tailored Bidirectional Vision Mamba
Abstract
Infrared--visible image fusion seeks to retain structures shared by the two modalities while preserving their complementary cues. The appropriate fusion rule, however, is not uniform across spatial locations or feature scales. Regions in which the modalities agree mainly require structure preservation, whereas regions with large cross-modal differences benefit from stronger complementary enhancement. We therefore introduce a coarse-to-fine framework with two coupled modules: an Uncertainty-Aware Dynamic Prompt Router (UDPR) and a Task-Tailored Bidirectional Vision Mamba (TT-BVM) block. UDPR predicts hierarchical uncertainty maps and uses them to route prompts across scales. TT-BVM uses the routed prompt and uncertainty map to separate common and discrepancy features, then models the two branches with dedicated bidirectional state-space blocks. The model is trained with an uncertainty-conditioned reconstruction loss and auxiliary terms that regularize routing and branch coordination. When trained only on MSRS, it obtains the best score in 16 of 20 metric entries across four public infrared--visible fusion benchmarks. The same model also performs competitively on three medical fusion benchmarks without retraining and improves semantic segmentation and object detection on fused images. These results support the uncertainty-guided design, and the mechanism analysis shows stable routing and branch behavior in the evaluated setting.