Towards Heterogeneous-Degradation-Robust Image Fusion with Controllable Generative Modulation.
TL;DR
DuS-DiFuse is proposed, a robust dual-stream latent diffusion framework composed of a diffusion fusion unit and a generative modulation unit that achieves leading fusion performance, exhibits strong robustness to heterogeneous degradations, generalizes well across fusion tasks, and supports effective controllable generative modulation.
Abstract
Enhancing degradation robustness is essential for deploying image fusion techniques in real-world dynamic scenes. However, most existing methods either handle a single degradation type or assume fixed multi-degradation settings, making them insufficient for dynamically heterogeneous and composition ally complex degradations in practice. Moreover, they often fail to recover the semantics of salient scene targets when these targets are degraded or missing, leading to weakened semantic representation and reduced target saliency. To address these challenges, we propose DuS-DiFuse, a robust dual-stream latent diffusion framework composed of a diffusion fusion unit and a generative modulation unit. In the diffusion fusion unit, we fine-tune a CLIP visual encoder on multi-source data to perceive degradation types and severities, and employ latent diffusion to uniformly model multi-type, cross-level degradations with varying parameters. A Groupwise Fusion Control Module (GFCM) is further embedded into the latent degradation-removal process, enabling joint modeling of dynamic degradation removal and multimodal information fusion. In the generative modulation unit, pretrained latent diffusion priors are used to remodulate the initial fusion results, enabling controllable semantic restoration and generative enhancement, thereby improving target saliency and overall visual quality. To preserve fine-grained details during latent-to image reconstruction, we introduce a Detail-Restoration Fidelity Module (DRFM), which constrains texture reconstruction by jointly leveraging multi-level skip features from multiple source images and enhances structural fidelity in the fused results. Extensive experiments on multiple fusion datasets demonstrate that DuS-DiFuse achieves leading fusion performance, exhibits strong robustness to heterogeneous degradations, generalizes well across fusion tasks, and supports effective controllable generative modulation.