一种红外热图像目标区域分割的深度学习算法

doi:10.19665/j.issn1001-2400.2019.04.015

摘要/Abstract

摘要：

为了解决复杂背景下红外热图像目标区域分割困难的问题,提出了一种利用全卷积网络和稠密条件随机场的深度学习分割算法。首先,利用全卷积网络进行像素级别特征提取,获得粗分割结果;然后,使用稠密条件随机场对粗分割结果进行上下文信息优化的精分割,最终实现目标区域的分割。将该算法应用于实际采集的太阳能板红外热图像数据集,五折交叉验证结果表明,该算法平均查准率为89.96%,平均查全率为94.55%,平均F₁指数为0.911 8,平均J指数为0.868 7。同时,最高查准率为93.35%,最高查全率为97.59%,最高F₁指数为0.956 2,最高J指数为0.912 5,均高于现有的主要算法。该算法耗时短且不需过多的人工干预,能实现复杂背景下红外热图像目标区域的有效分割。

关键词: 红外热图像, 分割, 全卷积网络, 稠密条件随机场

Abstract:

To tackle difficulties of the segmentation of the interested region in complex background, a deep-learning segmentation algorithm based on the fully convolutional network and the dense conditional random field is proposed. First, the fully convolutional network is leveraged for pixel-level feature extraction to obtain the coarse segmentation result. Then, the dense conditional random field which is used to optimize the context information is performed for detailed segmentation. Five-fold cross-validation experiments are carried out on an actually acquired infrared thermal image of the solar panel. Experimental results show that the proposed algorithm has an average precision rate of 89.96%, an average recall rate of 94.55%, an average F₁ index of 0.9118 and an average J index of 0.8687. At the same time, the algorithm achieves the best maximum precision rate of 93.35%, a maximum recall rate of 97.59%, a maximum F₁ index of 0.9562 and a maximum J index of 0.9125 compared with those by main existing algorithms. Moreover, this method takes less time and requires less manual interference. In conclusion, the proposed algorithm is capable of the segmentation of the interested region in the infrared thermal image effectively in the complex background.

Key words: infrared thermal image, segmentation, fully convolutional network, dense conditional random field

中图分类号:

TP391

朱莉,赵俊,傅应锴,张晶,沈惠,张守峰. 一种红外热图像目标区域分割的深度学习算法[J]. 西安电子科技大学学报, 2019, 46(4): 107-114.

ZHU Li,ZHAO Jun,FU Yingkai,ZHANG Jing,SHEN Hui,ZHANG Shoufeng. Deep learning algorithm for the segmentation of the interested region of an infrared thermal image[J]. Journal of Xidian University, 2019, 46(4): 107-114.

图/表 11

图1

图2

图3

表1

表2

五折交叉验证结果"

分割性能评价指标	交叉验证					均值
分割性能评价指标	1	2	3	4	5	均值
$P$ /%	90.18	88.09	93.35	87.78	90.39	89.96
$R$ /%	94.93	96.41	97.59	93.30	90.52	94.55
$F 1$	0.922 5	0.918 4	0.956 2	0.899 5	0.865 3	0.911 8
$J$	0.861 2	0.864 1	0.912 5	0.832 4	0.839 0	0.868 7

表2

表3

算法分割性能评价指标平均值对比"

算法	平均查准率 $P ˉ$ /%	平均查全率 $R ˉ$ /%	F₁指数平均值 $F ˉ 1$	J指数平均值 $J ˉ$
FCM	88.58	70.83	0.766 1	0.645 3
Chan-Vese	92.05	70.44	0.779 8	0.661 4
Otsu 文献[11,12]算法	82.31 93.06	70.35 90.28	0.740 6 0.914 4	0.607 0 0.851 5
文中算法	90.18	94.93	0.922 5	0.861 2

表3

表4

算法性能评价指标的T-test检验"

算法		$R$	$F 1$	$J$
FCM对比FCN-DCRF	$0.203$ ^#	$7.5 × 10 - 6$ ^***	$2.9 × 10 - 5$ ^***	$9.9 × 10 - 6$ ^***
Chan-Vese对比FCN-DCRF	$0.531$ ^#	$6.1 × 10 - 6$ ^***	$3.8 × 10 - 5$ ^***	$1.4 × 10 - 5$ ^***
Otsu对比FCN-DCRF	$1.6 × 10 - 3$ ^***	$4.9 × 10 - 6$ ^***	$8.8 × 10 - 7$ ^***	$2.1 × 10 - 7$ ^***

表4

图4

表5

算法性能评价指标对比"

图像	算法	$P$ /%	$R$ /%	$F 1$	$J$
1	FCM	99.14	58.93	0.739 2	0.586 3
1	Chan-Vese Otsu 文中算法	86.45 87.93 95.10	60.42 59.45 99.88	0.711 3 0.709 4 0.974 3	0.551 9 0.549 7 0.950 0
2	FCM	85.43	62.88	0.724 4	0.567 9
2	Chan-Vese Otsu 文中算法	85.14 81.95 84.02	62.74 62.99 73.08	0.722 4 0.712 3 0.781 7	0.565 5 0.553 2 0.641 6
3	FCM	91.88	40.55	0.562 7	0.391 5
3	Chan-Vese Otsu 文中算法	93.59 75.48 84.49	40.06 40.01 99.62	0.561 0 0.523 0 0.914 3	0.389 9 0.354 1 0.842 2
4	FCM	98.54	40.04	0.569 5	0.398 1
4	Chan-Vese Otsu 文中算法	98.73 92.83 85.77	40.63 42.43 99.97	0.575 6 0.582 4 0.923 4	0.404 1 0.410 8 0.857 7
5	FCM	99.21	90.11	0.944 4	0.894 7
5	Chan-Vese Otsu 文中算法	99.75 94.78 94.98	88.89 91.35 99.94	0.940 0 0.930 3 0.974 0	0.886 9 0.869 7 0.949 3

表5

图5

图6

参考文献 16

[1]	易翔, 王炳健 . 视觉显著性指导的红外与可见光图像融合算法[J]. 西安电子科技大学学报, 2019,46(1):1-6.
	YI Xiang, WANG Bingjian . Fusion of Infrared and Visual Images Guided by Visual Saliency[J]. Journal of Xidian University, 2019,46(1):1-6.
[2]	赖睿, 官俊涛, 徐昆然 , 等. 级联残差学习的红外图像非均匀性校正方法[J]. 西安电子科技大学学报, 2019,46(1):14-19.
	LAI Rui, GUAN Juntao, XU Kunran , et al. Cascade Residual Learning Method for Infrared Image Nonuniformity Correction[J]. Journal of Xidian University, 2019,46(1):14-19.
[3]	ZAITOUN N M, AQEL M J. Survey on Image Segmentation Techniques[C]// Procedia Computer Science: 65. Boston: Elsevier Academic Press, 2015: 797-806.
[4]	寇小明, 刘上乾, 洪鸣 , 等. 一种自适应红外图像增强技术[J]. 西安电子科技大学学报, 2009,36(6):1070-1074.
	KOU Xiaoming, LIU Shangqian, HONG Ming , et al. Approach for Adaptive Infrared Thermal Image Enhancement[J]. Journal of Xidian University, 2009,36(6):1070-1074.
[5]	HE F, GUO Y, GAO C . Human Segmentation of Infrared Image for Mobile Robot Search[J]. Multimedia Tools and Applications, 2018,77(9):10701-10714.
[6]	BAI X, CHEN Z, ZHANG Y , et al. Infrared Ship Target Segmentation Based on Spatial Information Improved FCM[J]. IEEE Transactions on Cybernetics, 2016,46(12):3259-3271.
[7]	ZHOU D, ZHOUO H. SHAO Y . An Improved Chan-Vese Model by Regional Fitting for Infrared Image Segmentation[J]. Infrared Physics and Technology, 2016 , 74:81-88.
[8]	JANSSENS O, VAN DE WALLE R, LOCCUFIER M, et al . Deep Learning for Infrared Thermal Image Based Machine Health Monitoring[J]. IEEE/ASME Transactions on Mechatronics, 2018,23(1):151-159.
[9]	LU T, LUONG A, HEIM S. et al. Intelligent Multi-Spectral IR Image Segmentation [C]//Proceedings of the SPIE: 10203. Bellingham: SPIE, 2017: 1020303.
[10]	DE MELO R H C, CONCI A, VASCONCELOS C N. . On the Use of Fully Convolutional Networks on Evaluation of Infrared Breast Image Segmentations [C]//Workshop de Informatica Médica. São Paulo: Sociedade Brasileira de Computação, 2017: 1897-1906.
[11]	朱莉, 张晶, 傅应锴 , 等. 基于多模态特征图融合的红外热图像目标区域提取算法[J]. 红外与毫米波学报, 2019,38(1):125-132.
	ZHU Li, ZHANG Jing, FU Yingkai , et al. Infrared Thermal Image ROI Extraction Algorithm Based on Fusion of Multi-Modal Feature Maps[J]. Journal of Infrared and Millimeter Waves, 2018,38(1):125-132.
[12]	SHEN H, ZHU L, HONGX G. et al. ROI Extraction Method of Infrared Thermal Image Based on GLCM Characteristic Imitate Gradient [C]//Proceedings of the 2017 2nd Chinese Conference on Computer Vision. Heidelberg: Springer Verlag, 2017: 192-205.
[13]	CHANDRA S, USUNIER N, KOKKINOS I. Dense and Low-Rank Gaussian CRFs Using Deep Embeddings [C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 5113-5122.
[14]	GAO X, MUNSON E, ABOUSLEMAN G P. et al. Automatic Solar Panel Recognition and Defect Detection Using Infrared Imaging [C]//Proceedings of the SPIE: 9476. Bellingham: SPIE, 2015: 1-9.
[15]	YUAN J, YANG H H L, OMITAOMU O A, et al. Large-Scale Solar Panel Mappingfrom Aerial Images Using Deep Convolutional Networks [C]//Proceedings of the 2016 IEEE International Conference on Big Data. Piscataway: IEEE, 2016: 2703-2708.
[16]	CARLSON J, MENICUCCI D, VOROBIEFF P , et al. Infrared Imaging Method for Flyby Assessment of Solar Thermal Panel Operation in Field Settings[J]. Applied Thermal Engineering, 2014,70(1):163-171.