西安电子科技大学学报 ›› 2020, Vol. 47 ›› Issue (1): 120-127.doi: 10.19665/j.issn1001-2400.2020.01.017

• • 上一篇    下一篇

一种注意力机制的多波段图像特征级融合方法

杨晓莉,蔺素珍()   

  1. 中北大学 大数据学院, 山西 太原 030051
  • 收稿日期:2019-07-12 出版日期:2020-02-20 发布日期:2020-03-19
  • 通讯作者: 蔺素珍
  • 作者简介:杨晓莉(1994—),女,中北大学硕士研究生,E-mail:2351257955@qq.com
  • 基金资助:
    山西省应用基础研究项目(201701D121062);中北大学第十五届科技立项项目(2018514)

Method for multi-band image feature-level fusion based on the attention mechanism

YANG Xiaoli,LIN Suzhen()   

  1. School of Data Science and Technology, North University of China, Taiyuan 030051, China
  • Received:2019-07-12 Online:2020-02-20 Published:2020-03-19
  • Contact: Suzhen LIN

摘要:

针对多波段同步融合图像普遍存在的清晰度不高、图像细节不丰富的问题,提出一种基于注意力机制生成对抗网络的图像特征级融合方法。首先,利用多波段特征图与其均值的差值构建注意力权重图,通过特征图与注意力权重图的点乘和相加获得特征增强图,以此构建特征增强模块;其次,设计特征级融合模块,将多波段特征增强图连接,通过归一化、上采样、卷积等操作重构融合图像;最后,将特征增强模块和特征融合模块级联建立生成器,并以VGG-16作为判别器构建生成对抗网络,以实现多波段图像端到端融合。实验结果表明,与当前经典的融合方法相比,所提出方法的平均梯度最为突出,验证了该方法的有效性。

关键词: 图像融合, 深度学习, 多波段图像, 特征级融合, 注意力机制, 生成对抗网络

Abstract:

Aiming at the low definition and poor details of synchronous multi-band image fusion, a novel method based on attention generative adversarial networks is proposed. First, the attention weight map is constructed using the difference between the multi-band feature map and its mean, then the feature enhancement map is obtained by the point multiplication and addition of the feature map and the attention weight map to construct the feature enhancement module. Second, the feature-level fusion module is designed, which connects the multi-band feature enhancement map and reconstructs the fused image through normalization, upsampling, convolution, etc. Finally, the feature enhancement module and the feature-level fusion module are cascaded to build the generator, and the VGG-16 is used as a discriminator to establish a Generative Adversarial Network, thereby implementing multi-band image end-to-end fusion. Experimental results show that the proposed fusion method can lead to the most prominent average gradient compared with classical fusion methods, and that the effectiveness of the proposed method is verified.

Key words: image fusion, deep learning, multi-band image, feature-level fusion, attention, generative adversarial networks

中图分类号: 

  • TP391