一种改进Unet网络的遥感影像分割算法

doi:10.19665/j.issn1001-2400.2022.06.009

摘要/Abstract

摘要：

现有的遥感影像分割算法将边缘信息和语义信息进行简单的结合,往往不能确保语义建模的整体改进。为解决此类问题,提出了基于改进Unet网络的遥感影像分割算法。该算法在基础编码器的基础上添加了边缘提取模块,此模块融合骨干网络所提取的语义特征信息以及由输入图像经过Canny算子和膨胀数学形态学操作获得的边缘特征信息,更好地学习了遥感影像的边缘。为了进一步获取遥感影像全局信息以提高分割精度,提出了边缘引导上下文聚合模块。该模块通过捕获边缘区域的像素和物体内部像素之间的长距离依赖关系,通过聚合上下文信息而加强类内一致性。在"天智杯 "人工智能挑战赛数据集的测试下,改进后的模型总体准确度达到84.5%,平均交并比达到68.6%,精度与经典Unet模型相比分别提高了5.3%和9.2%。改进后的模型在ISPRS Vaihingen和Potsdam基准数据集上总体准确度分别达到了91.2%和91.6%,更适于精确的遥感影像分割。

关键词: 语义分割, Canny算子, 数学形态学操作, 深度学习, 图像处理

Abstract:

Existing remote sensing image segmentation algorithms simply combine edge information with semantic information,they often fail to ensure an overall improvement in semantic modelling.To solve such problems,an improved remote sensing image segmentation algorithm using Unet networks is proposed.The improved algorithm adds an edge extraction module to the base encoder module.The module fuses the semantic feature information of the backbone network and the boundary feature information obtained from the input image by Canny operator and dilated mathematical morphological operations to learn the edges of remote sensing image.To further acquire global information of remote sensing images for improving segmentation accuracy,an edge-guided context aggregation module is proposed.This module enhances the intra-class consistency by capturing the long-distance dependencies between pixels in the boundary region and pixels inside the object,and then aggregates the contextual information.Under the test of the "Tianzhi Cup" AI Challenge dataset,the overall accuracy of the improved model reached about 84.5% and the average intersection ratio reached about 68.6%,with an accuracy improvement of 5.3% and 9.2% respectively compared with the Unet model.The improved model achieved an overall accuracy of 91.2% and 91.6% on the ISPRS Vaihingen and Potsdam benchmark datasets respectively,making it more suitable for accurate remote sensing image segmentation.

Key words: semantic segmentation, Canny, mathematical morphological operations, deep learning, image processing

中图分类号:

TP391.4

李娇娇, 刘志强, 宋锐, 李云松. 一种改进Unet网络的遥感影像分割算法[J]. 西安电子科技大学学报, 2022, 49(6): 67-75.

LI Jiaojiao, LIU Zhiqiang, SONG Rui, LI Yunsong. Algorithm for segmentation of remote sensing imagery using the improved Unet[J]. Journal of Xidian University, 2022, 49(6): 67-75.

图/表 9

图1

表1

图2

表2

图3

表3

图4

表4

图5

参考文献 25

[1]	LONG J, SHELHAMER E, DARREL T. Fully Convolutional Networks for Semantic Segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2015:3431-3440.
[2]	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Deeplab:Semantic Image Segmentation with Deep Convolutional Nets,Atrous Convolution,and Fully Connected Crfs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4):834-848. doi: 10.1109/TPAMI.2017.2699184
[3]	ZHAO H S, SHI J P, QI X J, et al. Pyramid Scene Parsing Network[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:2881-2890.
[4]	ZHANG H, DANA K, SHI J P, et al. Context Encoding for Semantic Segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:7151-7160.
[5]	WANG X L, GIRSHICK R, GUPTA A, et al. Non-Local Neural Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:7794-7803.
[6]	ZHOU Q, YANG W, GAO G W, et al. Multi-Scale Deep Context Convolutional Neural Networks for Semantic Segmentation[J]. World Wide Web, 2019, 22(2):555-570. doi: 10.1007/s11280-018-0556-3
[7]	ZHOU Q, WANG Y, LIU J, et al. An Open-Source Project for Real-Time Image Semantic Segmentation[J]. Science.China Information Sciene, 2019, 62(12):227101.
[8]	YU C Q, WANG J B, PENG C, et al. Learning A Discriminative Feature Network for Semantic Segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:1857-1866.
[9]	ZHANG L, LI X T, AMAB A, et al. Dual Graph Convolutional Network for Semantic Segmentation[C]// Proceedings of the British Machine Vision Conference.Heidelberg:Springer, 2019:2111-2123.
[10]	ZHANG L, XU D, AMAB A, et al. Dynamic Graph Message Passing Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2020:3726-3735.
[11]	CHEN L C, GONTIJO R, CHENG B, et al. Naive-Student:Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation[C]// Proceedings of the European Conference on Computer Vision.Heidelberg:Springer, 2020:695-714.
[12]	BERTASIUS G, SHI J B, TORRESANI L. Semantic Segmentation with Boundary Neural Fields[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:3602-3610.
[13]	CHEN L C, BARRON J, PAPANDREOU G. Semantic Image Segmentation with Task-Specific Edge Detection Using Cnns and a Discriminatively Trained Domain Transform[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:4545-4554.
[14]	ZHEN M M, WANG J L, ZHOU L, et al. Joint Semantic Segmentation and Boundary Detection Using Iterative Pyramid Contexts[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2020:13666-13675.
[15]	LI X T, LI X, ZHANG L, et al. Improving Semantic Segmentation Via Decoupled Body and Edge Supervision[C]// Proceedings of the European Conference on Computer Vision.Heidelberg:Springer, 2020:435-452.
[16]	YUANY H, XIEJ Y, CHENX L, et al. Segfix:Model-Agnostic Boundary Refinement for Segmentation[C]// Proceedings of the European Conference on Computer Vision.Heidelberg:Springer, 2020:489-506.
[17]	DING H H, JIANG X D, LIU A Q, et al. Boundary-Aware Feature Propagation for Scene Segmentation[C]// Proceedings of the IEEE International Conference on Computer Vision.Piscataway:IEEE, 2019:6819-6829.
[18]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is All You Need[J]. Advances in Neural Information Processing Systems, 2017, 30(1):5998-6008.
[19]	CHEN L C, YANG Y, WANG J, et al. Attention to Scale:Scale-Aware Semantic Image Segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:3640-3649.
[20]	FU J, LIU J, TIAN H, et al. Dual Attention Network for Scene Segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:3146-3154.
[21]	HUANG Z L, WANGX G, HUANG L C, et al. CCnet:Criss-Cross Attention for Semantic Segmentation[C]// Proceedings of the IEEE International Conference on Computer Vision.Piscataway:IEEE, 2019:603-612.
[22]	YIN M H, YAO Z L, CAO Y, et al. Disentangled Non-Local Neural Networks[C]// Proceedings of the European Conference on Computer Vision.Heidelberg:Springer, 2020:191-207.
[23]	周鹏, 杨军. 采用神经网络架构搜索的遥感影像分割方法[J]. 西安电子科技大学学报, 2021, 48(5):47-57.
	ZHOU Peng, YANG Jun. Semantic Segmentation of Remote Sensing Images Based on Neural Architecture Search[J]. Journal of Xidian University, 2021, 48(5):47-57.
[24]	汪梓艺, 苏育挺, 刘艳艳. 一种改进DeeplabV3网络的烟雾分割算法[J]. 西安电子科技大学学报, 2019, 46(6):52-59.
	WANG Ziyi, SU Yuting, LIU Yanyan, et al. Algorithm for Segmentation of Smoke Using the Improved DeeplabV3 Network[J]. Journal of Xidian University, 2019, 46(6):52-59.
[25]	党吉圣, 杨军. 多特征融合的三维模型识别与分割[J]. 西安电子科技大学学报, 2020, 47(4):149-157.
	DANG Jisheng, YANG Jun. 3D Model Recognition and Segmentation Based on Multi-Feature Fusion[J]. Journal of Xidian University, 2020, 47(4):149-157.

方法名称	a	b	c	d
边缘提取模块			√	√
边缘提取模块(不加Canny分支)		√
边缘引导上下文聚合模块		√	√	√
损失函数L_edge				√
R_mIoU/%	59.42	60.03	60.53	61.76

方法名称	农场	道路	水源	植被	R_OA	R_mIoU
FCN	58.9	24.6	92.4	74.8	83.0	62.7
Unet	60.0	20.5	92.5	64.4	79.2	59.4
SegNet	59.1	28.6	92.6	73.0	82.4	63.3
PSPNet	64.5	34.9	93.9	72.7	83.4	66.5
DeeplabV3+	65.2	34.9	93.8	73.4	83.7	66.8
Ours	65.4	40.8	93.7	74.6	84.5	68.6

方法名称	不可渗透表面	建筑物	低矮植被	树木	汽车	背景	R_OA	F₁均值
SVL_3	86.6	91.0	77.0	85.0	55.6	58.9	84.8	79.0
DST_2	90.5	93.7	83.4	89.2	72.6	61.2	89.1	85.9
UZ_1	89.2	92.5	81.6	86.9	57.3	58.6	87.3	81.5
RIT_L7	90.1	93.2	81.4	87.2	72.0	63.4	87.8	84.8
ONE_7	91.0	94.5	84.4	89.9	77.8	71.9	89.8	87.5
ADL_3	89.5	93.2	82.3	88.2	63.3	69.6	88.0	83.3
DLR_10	92.3	95.2	84.1	90.0	79.3	79.3	90.3	88.2
CASIA2	93.2	96.0	84.7	89.9	86.7	84.7	91.1	90.1
BKHN10	92.9	96.0	84.6	89.8	88.8	81.1	91.0	90.4
TreeUNet	92.5	94.9	83.6	89.6	85.9	82.6	90.4	89.3
文中	93.4	95.9	84.6	90.2	89.6	85.9	91.2	90.6

方法名称	不可渗透表面	建筑物	低矮植被	树木	汽车	杂物	R_OA	F₁均值
SVL_1	83.5	91.7	72.2	63.2	62.2	69.3	77.8	74.6
DST_5	92.5	96.4	86.7	88.0	94.7	78.4	90.3	91.7
UZ_1	89.3	95.4	81.8	80.5	86.5	79.7	85.8	86.7
RIT_L7	91.2	94.6	85.1	85.1	92.8	83.2	88.4	89.8
SWJ_2	94.4	97.4	87.8	87.6	94.7	82.1	91.7	92.4
CASIA2	93.3	97.0	87.7	88.4	96.2	83.0	91.1	92.5
TreeUNet	93.1	97.3	86.8	87.1	95.8	82.9	90.7	92.0
文中	93.6	97.2	89.0	89.6	96.3	86.4	91.6	93.1