MEF-TBN: Multi-exposure image fusion approach based on a three-branch network for feature extraction and color enhancement

Abstract

High dynamic range (HDR) imaging is crucial in fields like photography and medical image processing. Traditional multi-exposure fusion (MEF) methods often struggle to simultaneously preserve local and global features as well as chrominance information. To address this issue, we propose MEF-TBN, a three-branch network consisting of a context aggregation attention network (CAAN) branch for local feature extraction, a transformer branch for global dependency modeling, and a color enhancement branch for natural color restoration. Specifically, the CAAN and transformer branches generate low-resolution weight maps based on local and global features of luminance components, and a merging module is proposed to refine and enhance the local and global features with several merger blocks, and then estimate high-resolution weights by adopting the guided filtering for upsampling operation. Meanwhile, the color enhancement branch is capable of learning the inherent correlation between luminance and chrominance components. Experimental results show that compared with 9 representative MEF methods, our method achieves optimal performance on most image feature-based metrics including AG, EI and SF, as well as all human perception-oriented metrics such as QCB, VIF and NIQE, and ranks second in the image structural similarity-based metric MEF-SSIM. In particular, compared with the state-of-the-art transformer-based method SwinFusion, our method achieves average performance improvements of 11.5%, 19.8%, 2.2% and 1.6% on QCB, VIF, NIQE and MEF-SSIM metrics, respectively. Both quantitative and qualitative experiments conducted on three MEF datasets with different input exposure numbers demonstrate that the proposed method achieves superior performance than the other methods.