Title
KT-GAN: Knowledge-Transfer Generative Adversarial Network for Text-to-Image Synthesis
Document Type
Article
Publication Date
1-1-2021
Abstract
This paper presents a new framework, Knowledge-Transfer Generative Adversarial Network (KT-GAN), for fine-grained text-to-image generation. We introduce two novel mechanisms: an Alternate Attention-Transfer Mechanism (AATM) and a Semantic Distillation Mechanism (SDM), to help generator better bridge the cross-domain gap between text and image. The AATM updates word attention weights and attention weights of image sub-regions alternately, to progressively highlight important word information and enrich details of synthesized images. The SDM uses the image encoder trained in the Image-to-Image task to guide training of the text encoder in the Text-to-Image task, for generating better text features and higher-quality images. With extensive experimental validation on two public datasets, our KT-GAN outperforms the baseline method significantly, and also achieves the competive results over different evaluation metrics.
Publication Source (Journal or Book title)
IEEE Transactions on Image Processing
First Page
1275
Last Page
1290
Recommended Citation
Tan, H., Liu, X., Liu, M., Yin, B., & Li, X. (2021). KT-GAN: Knowledge-Transfer Generative Adversarial Network for Text-to-Image Synthesis. IEEE Transactions on Image Processing, 30, 1275-1290. https://doi.org/10.1109/TIP.2020.3026728