Infrared and visible image fusion based on nested model decomposition for autonomous driving

Abstract

To address the problem that visible images degrade severely under nighttime, tunnel, strong backlight, rain and fog conditions, while infrared images can highlight thermal targets but lack detailed texture, this paper proposes an infrared and visible image fusion method based on nested model decomposition. First, latent low-rank representation (LatLRR) decomposes source images into a base part and a visual saliency part. Second, weighted least squares (WLS) filtering further decomposes the base part into low-frequency structural sub-bands and high-frequency detail sub-bands. A weighted average strategy is adopted for low-frequency components to preserve global structures. For high-frequency components, a VGG- 19 multi-layer activation fusion strategy is introduced to retain edge and texture details. Meanwhile, an improved visual saliency map control law is used to adaptively enhance pedestrians, vehicles and other thermal targets. Finally, two-stage reconstruction produces the final fused image. Experimental results on representative autonomous-driving infrared-visible image pairs show that the proposed method generates fused images with prominent thermal targets, rich details and low noise artifacts.