Visible and infrared image fusion based on deep learning

Abstract

Feature extraction is a common point at which current image fusion approaches fall short in establishing relationships between local and global information. The result is merged photos of low quality. This research presents an interactive transformer-based infrared and visible image fusion network to solve this problem. In order to begin, the network extracts shallow features from the input infrared and visible pictures using a residual dense block. The next step is the creation of an interactive transformer module. It takes the input photos and uses both global and local properties to build relationships. To get even more features out of the original photos, we employ two matching interactive transformer modules. The feature maps that the transformer modules extracted, which are infrared and visible, are then fused using a fusion module. At last, in order to achieve infrared and visible picture fusion, the final fused image is generated using an image reconstruction network. In order to test the method, the TNO dataset is used. According to the findings of the experiments, our method's fused pictures provide more information about infrared heat radiation and visible textures. The fusion performance of our technique is better than that of competing methods.