The proposed UniDiffFusion framework is trained in a progressive manner to decouple fusion learning, degradation-aware restoration, and application-specific adaptation, thereby reducing interference among heterogeneous objectives.
Abstract
General image fusion aims to integrate complementary information from multiple source images, but existing methods often rely on task-specific models and struggle to maintain robust performance under diverse degradation conditions. In this paper, we propose UniDiffFusion, a unified diffusion framework for multi-task and degradation-robust image fusion. UniDiffFusion leverages the strong generative prior of a pretrained diffusion model to establish a shared fusion backbone across heterogeneous fusion tasks, while introducing task- and degradation-aware conditional adaptation to accommodate their distinct information-selection requirements. Specifically, we employ task prompt modulation to progressively adapt the shared diffusion representations to different fusion objectives, and develop a degradation prompt router to dynamically retrieve degradation-aware priors and restore corrupted source features before fusion. Furthermore, an application prompt bank is introduced to incorporate task-oriented semantic guidance for downstream applications, such as object detection and semantic segmentation, without altering the shared fusion and restoration pathways. The proposed framework is trained in a progressive manner to decouple fusion learning, degradation-aware restoration, and application-specific adaptation, thereby reducing interference among heterogeneous objectives. Extensive experiments on visible-infrared, multi-exposure, and multi-focus image fusion demonstrate that UniDiffFusion achieves superior fusion quality and robustness under both clean and degraded conditions. Moreover, UniDiffFusion consistently improves downstream detection and semantic segmentation performance, demonstrating its effectiveness as a unified diffusion framework for both perceptual fusion and task-oriented vision.
Domain adaptive object detection (DAOD) aims to reduce performance degradation due to distribution shifts between training (source) and testing (target) datasets. However, owing to insufficient modeling of fine-grained features and limited instance-level alignment, existing approaches become less effective under comple...
Zhen-Hua Yu, Jia-Hao Zhang, Wen-Chao Zhang et al.· 0 citations
Diff-RF explores the intrinsic coupling between registration-fusion and information restoration in the degradation conditions, enabling high-quality fusion of unregistered images under complex degradation conditions.
Xun-Peng Yi, Zai-Xi Du, Qing-Long Yan et al.· arXiv.org· 0 citations
This work proposes EdgeAttnSwin, a framework that reformulates fusion as a three-stage progressive optimization process with explicit causal dependencies, and demonstrates that this method outperforms state-of-the-art approaches on infrared-visible and medical image fusion benchmarks.
A novel diffusion framework that employs interactive modeling of determinism and stochasticity inherent in multiple image degradation processes, enabling it to handle diverse IR scenarios and demonstrating excellent generality and adaptability.
Sha Luo, Si-Yu Peng, Da-Wei Zhao et al.· The Visual Computer· 0 citations
V VesperMUSE is introduced, a unified multi-modal framework designed to achieve robust image semantic segmentation across diverse illumination conditions and successfully recovers missing visual cues, providing a flexible, state-of-the-art (SOTA) solution for machine perception in extreme and unstructured real-world en...
Raul Balmez· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.