视觉显著性指导的红外与可见光图像融合算法

doi:10.19665/j.issn1001-2400.2019.01.005

摘要/Abstract

摘要：

为了获取适合人眼观测的高质量红外与可见光融合图像,提出了一种基于视觉显著性指导的红外与可见光图像融合算法。首先,利用改进的流形排序法分别检测红外与可见光图像的视觉显著性区域;然后,采用非下采样轮廓波变换对红外和可见光图像进行多尺度、多方向分解,从而获取各自低频子带和高频子带,并将视觉显著性的检测结果用于指导分配低频子带的融合权重,即依据显著度大小赋予不同的权值,而高频子带的融合则依据局部标准差准则赋值;最后,通过非下采样轮廓波逆变换获得融合图像。实验结果表明:这种算法不仅可以保全可见光图像中的细节信息,而且能够精确地突显出红外目标信息,具有较好的视觉效果, 增强了红外与可见光复合前视系统的识别性能。

关键词: 图像融合, 红外与可见光图像, 非下采样轮廓波, 视觉显著性

Abstract:

To obtain a high quality fused image consistent with characteristics of human vision, a novel image fusion method for infrared and visual images guided by visual saliency is proposed. First of all, for the given infrared and visible images, the modified Manifold Ranking algorithm is utilized to extract their visual salient areas respectively. Then, source images are decomposed in different scales and directions by Non-subsampled Contourlet Transform to obtain low frequency information and high frequency information. And results of visual saliency detection are used to guide the fusion rule of low frequency subband coefficients. Besides, the high frequency subband coefficients are fused owing to the local standard deviation criterion. Finally, the fused image is obtained by performing inverse Non-subsampled Contourlet Transform. Experimental results demonstrate that the proposed algorithm can not only assure the final fused images with clear detail information, but also highlight the infrared objects accurately, which presents a good vision effect and effectively enhances recognition probability of infrared and visible compound systems.

Key words: image fusion, infrared and visible images, non-subsampled contourlet, visual saliency

中图分类号:

TP391

易翔,王炳健. 视觉显著性指导的红外与可见光图像融合算法[J]. 西安电子科技大学学报, 2019, 46(1): 27-32.

YI Xiang,WANG Bingjian. Fusion of infrared and visual images guided by visual saliency[J]. Journal of Xidian University, 2019, 46(1): 27-32.

图/表 7

图1

图2

图3

图4

图5

表1

表2

参考文献 17

[1]	LIU C H, QI Y, DING W R . Infrared and Visible Image Fusion Method Based on Saliency Detection in Sparse Domain[J]. Infrared Physics and Technology, 2017,83:94-102. doi: 10.1016/j.infrared.2017.04.018
[2]	霍冠英, 李庆武, 石丹 . 一种邻域一致性的NSCT域多传感器图像融合算法[J]. 西安电子科技大学学报, 2010,37(4):770-776. doi: 10.3969/j.issn.1001-2400.2010.04.033
	HUO Guanying, LI Qingwu, SHI Dan . Multi-sensor Image Fusion Algorithm Considering Neighborhood Consistency in the Nonsubsampled Contourlet Transform Domain[J]. Journal of Xidian University, 2010,37(4):770-776. doi: 10.3969/j.issn.1001-2400.2010.04.033
[3]	余美晨, 孙玉秋, 王超 . 基于拉普拉斯金字塔的图像融合算法研究[J]. 长江大学学报(自然科学版), 2016,13(34):21-26.
	YU Meichen, SUN Yuqiu, WANG Chao . Image Fusion Algorithm Based on Laplacian Pyramid[J]. Journal of Yangtze University (Natural Science Edition), 2016,13(34):21-26.
[4]	ZUO Y J, LIU J, BAI G , et al. Airborne Infrared and Visible Image Fusion Combined with Region Segmentation[J]. Sensors, 2017,17(5):1127. doi: 10.3390/s17051127 pmid: 5470803
[5]	KONG W, LEI Y, Ni X . Fusion Technique for Grey-scale Visible Light and Infrared Images Based on Non-subsampled Contourlet Transform and Intensity-hue-saturation Transform[J]. IET Signal Processing, 2011,5(1):75-80. doi: 10.1049/iet-spr.2009.0263
[6]	CUNHA A L D, ZHOU J, DO M N . The Nonsubsampled Contourlet Transform: Theory, Design, and Applications[J]. IEEE Transactions on Image Processing, 2006,15(10):3089-3101. doi: 10.1109/TIP.2006.877507 pmid: 17022272
[7]	LIAO J W, HUANG J, WANG Z, et al. Fusion of Visible Image and Infrared Image Based on Contourlet Transform and Improved Spatial Frequence [C]//Proceedings of the 2014 2nd International Conference on Information Technology and Electronic Commerce. Piscataway: IEEE, 2014: 322-325.
[8]	LIU H X, ZHU T H, ZHAO J J . Infrared and Visible Image Fusion Based on Region of Interest Detection and Nonsubsampled Contourlet Transform[J]. Journal of Shanghai Jiaotong University, 2013,18(5):526-534. doi: 10.1007/s12204-013-1437-7
[9]	FU Z Z, WANG X, L I X F , et al. Infrared and Visible Image Fusion Based on Visual Saliency and NSCT[J]. Journal of University of Electronic Science and Technology of China, 2017,46(2):357-362. doi: 10.3969/j.issn.1001-0548.2017.02.007
[10]	CUI G M, FENG H, XU Z , et al. Detail Preserved Fusion of Visible and Infrared Images Using Regional Saliency Extraction and Multi-scale Image Decomposition[J]. Optics Communications, 2015,341:199-209. doi: 10.1016/j.optcom.2014.12.032
[11]	YANG C, ZHANG L, LH H, et al. Saliency Detection via Graph-based Manifold Ranking [C]//Proceedings of the 2013 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2013: 3166-3173.
[12]	ACHANTA R, SHAJI A, SMITH K , et al. SLIC Superpixels Compared to State-of-the-art Superpixel Methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012,34(11):2274-2281. doi: 10.1109/TPAMI.2012.120 pmid: 22641706
[13]	XIANG T Z, GAO R, YAN L , et al. Region Feature Based Multi-scale Fusion Method for Thermal Infrared and Visible Images[J]. Geomatics and Information Science of Wuhan University, 2017,42(7):911-917. doi: 10.13203/j.whugis20141007
[14]	HUANG Y, BI D, WU D . Infrared and Visible Image Fusion Based on Different Constraints in the Non-subsampled Shearlet Transform Domain[J]. Sensors, 2018,18(4):1169. doi: 10.3390/s18041169
[15]	PARAMANDANDHAM N, RAJENDIRAN K. A Simple and Efficient Image Fusion Algorithm Based on Standard Deviation in Wavelet Domain [C]//Proceedings of the 2016 IEEE International Conference on Wireless Communications, Signal Processing and Networking. Lausanne: Presses Polytechniques Et Universitaires Romandes, 2016: 2207-2211.
[16]	MA J L, ZHOU Z, WANG B , et al. Infrared and Visible Image Fusion Based on Visual Saliency Map and Weighted Least Square Optimization[J]. Infrared Physics and Technology, 2017,82:8-17. doi: 10.1016/j.infrared.2017.02.005
[17]	MA J Y, MA Y, LI C . Infrared and Visible Image Fusion Methods and Applications: a Survey[J]. Information Fusion, 2017,45:153-178. doi: 10.1016/j.inffus.2018.02.004

算法	EPI	FD	SE	SF	SSIM
F_NSCT	38.0457	5.7113	6.2977	10.1452	0.7018
F_RS	40.5689	5.9047	6.5027	10.0025	0.7569
F_SR	38.1454	5.9587	6.1689	10.0693	0.7125
F_IT	38.2381	5.7436	6.3059	10.1978	0.7153
F_MR	41.9066	5.9615	6.7566	10.7781	0.7391
文中算法	46.2144	6.2580	7.2836	11.5546	0.7530

算法	EPI	FD	SE	SF	SSIM
F_NSCT	28.4873	3.3918	6.6066	6.3540	0.8070
F_RS	24.0299	2.7364	6.4061	5.0177	0.8297
F_SR	27.9010	3.1738	7.0383	5.7975	0.8050
F_IT	26.1412	2.9763	7.0113	5.4485	0.8101
F_MR	25.5382	2.9906	6.6588	5.5298	0.8165
文中算法	31.2861	3.7087	7.1592	6.9202	0.8186