Abstract:

Aiming at the low definition and poor details of synchronous multi-band image fusion, a novel method based on attention generative adversarial networks is proposed. First, the attention weight map is constructed using the difference between the multi-band feature map and its mean, then the feature enhancement map is obtained by the point multiplication and addition of the feature map and the attention weight map to construct the feature enhancement module. Second, the feature-level fusion module is designed, which connects the multi-band feature enhancement map and reconstructs the fused image through normalization, upsampling, convolution, etc. Finally, the feature enhancement module and the feature-level fusion module are cascaded to build the generator, and the VGG-16 is used as a discriminator to establish a Generative Adversarial Network, thereby implementing multi-band image end-to-end fusion. Experimental results show that the proposed fusion method can lead to the most prominent average gradient compared with classical fusion methods, and that the effectiveness of the proposed method is verified.

Key words: image fusion, deep learning, multi-band image, feature-level fusion, attention, generative adversarial networks