FE-GAN: feature enhancement and fine-grained interaction for text-to-image synthesis

Abstract

In the task of text-to-image synthesis, it is challenging to generate semantically consistent and high-fidelity images from given text descriptions. To address the issues of semantic inconsistency between text and images and the lack of realism in generated image details, this paper proposes a GAN-based method for generating images from text. Specifically, by introducing multi-kernel convolution into the generator and replacing standard convolutions with multi-layer nested convolutions combined with dynamic gating weighting, the model is encouraged to focus on crucial detailed features, thereby improving the fine-grained quality of generated images. Then, we design a feature enhancement block that models multi-dimensional feature interactions via channel, spatial height, and spatial width branches, which effectively enhances feature representation and produces more photo-realistic images. Furthermore, we propose a fine-grained interaction mechanism that utilizes a low-rank correlation matrix to capture the dependencies between global and local information at different granularity levels, achieving fine-grained channel enhancement and significantly improving text-image alignment accuracy. Moreover, the proposed model maintains the advantages of efficient generation and a smooth latent space from the GAN paradigm. Finally, comprehensive experiments are carried out on the CUB and COCO benchmarks. Quantitative results illustrate that our approach achieves 11.48 FID and 5.23 IS on the CUB dataset, and 17.46 FID and 36.03 IS on the COCO dataset, respectively.