多特征融合的三维模型识别与分割

doi:10.19665/j.issn1001-2400.2020.04.020

摘要/Abstract

摘要：

针对现有的基于深度学习的三维模型识别与分割方法忽略了三维模型高级全局单点特征和低级局部几何特征之间的关系而导致识别效果不佳的问题,提出了一种多特征融合的三维模型识别与分割方法。首先,通过加深卷积核的宽度和网络的深度构建全局单点网络以提取具有高级语义识别能力的全局单点特征;其次,通过构建注意力融合层学习全局单点特征和局部几何特征的隐含关系来充分挖掘更能表征模型类别的细粒度几何特征;最后,将全局单点特征和细粒度几何特征进一步融合,达到优势互补、增强特征丰富性的目的。分别在三维模型识别数据集ModelNet40、ModelNet10和分割数据集ShapeNet Parts、S3DIS、vKITTI上进行了实验验证,并与当前主流识别算法进行了对比,表明该算法不仅有更高的识别和分割准确率,而且具有更强的鲁棒性。

关键词: 三维点云, 目标识别, 语义分割, 注意力融合, 深度学习, 卷积神经网络

Abstract:

Current methods focusing on 3D model recognition and segmentation have to some extent ignored the relationship between the high-level global single-point features and the low-level local geometric features of those models, resulting in poor recognition results. A multi-feature fusion approach which takes into consideration the aforementioned ignored relationship is proposed. First, a global single-point network is established to extract the global single-point features with high-level semantic recognition ability by increasing both the width of convolution kernel and the depth of the network. Second, an attentional fusion layer is constructed to learn the implicit relationship between global single-point features and local geometric features to fully explore the fine-grained geometric features that can better represent model categories. Finally, the global single-point features and fine-grained geometric features are further fused to achieve the complementation of advantages and enhance the feature richness. Experimental verification is carried out on the 3D model recognition datasets ModelNet40, ModelNet10 and segmentation datasets ShapeNet Parts, S3DIS, vKITTI, respectively, and comparison with current mainstream recognition algorithms shows that the proposed algorithm not only has higher recognition and segmentation accuracy, but also has stronger robustness.

Key words: 3D point cloud, object recognition, semantic segmentation, attention fusion, deep learning, convolutional neural networks

中图分类号:

TP391

党吉圣,杨军. 多特征融合的三维模型识别与分割[J]. 西安电子科技大学学报, 2020, 47(4): 149-157.

DANG Jisheng,YANG Jun. 3D model recognition and segmentation based on multi-feature fusion[J]. Journal of Xidian University, 2020, 47(4): 149-157.

图/表 10

图1

图2

表1

图3

表2

图4

表3

图5

图6

图7

参考文献 26

[1]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet Classification with Deep Convolutional Neural Networks[C]// Proceedings of the 2012 25th International Conference on Neural Information Processing Systems: 1. Vancouver: Neural Information Processing Systems Foundation, 2012: 1097-1105.
[2]	张金刚, 方圆, 袁豪, 等. 一种识别表情序列的卷积神经网络[J]. 西安电子科技大学学报, 2018,45(1):150-155.
	ZHANG Jingang, FANG Yuan, YUAN Hao, et al. Multiple Convolutional Neural Networks for Facial Expression Sequence Recognition[J]. Journal of Xidian University, 2018,45(1):150-155.
[3]	朱苏雅, 杜建超, 李云松, 等. 采用U-Net 卷积网络的桥梁裂缝检测方法[J]. 西安电子科技大学学报, 2019,6(4 ): 35-42.
	ZHU Suya, Du Jianchao, LI Yunsong, et al. Method for Bridge Crack Detection Based on the U-Net Convolutional Network[J]. Journal of Xidian University, 2019,6(4):35-42.
[4]	刘道华, 崔玉爽, 赵岩松, 等. 一种改进卷积神经网络的教学图像检索方法[J]. 西安电子科技大学学报, 2019(3):52-58.
	LIU Daohua, CUI Yushuang, ZHAO Yansong, et al. Method for Retrieving the Teaching Image Based on the Improved Convolutional Neural Networks[J]. Journal of Xidian University, 2019(3):52-58.
[5]	WU Z, SONG S, KHOSLA A, et al. 3D Shapenets: a Deep Representation for Volumetric Shapes[C]// Proceedings of the 2015 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2015: 1912-1920.
[6]	MATURANA D, SCHERER S. Voxnet: A 3D Convolutional Neural Network for Real-time Object Recognition[C]// Proceedings of the 2015 IEEE International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2015: 922-928.
[7]	杨军, 王亦民. 基于深度卷积神经网络的三维模型识别[J]. 重庆邮电大学学报, 2019,31(2):253-260.
	YANG Jun, WANG Yimin. 3D Model Recognition and Classification Based on Deep Convolution Neural Network[J]. Journal of Chongqing University of Posts and Telecommunications, 2019,31(2):253-260.
[8]	杨军, 王顺, 周鹏. 基于深度体素卷积神经网络的三维模型识别分类[J]. 光学学报, 2019,39(4):314-324.
	YANG Jun, WANG Shun, ZHOU Peng. 3D Model Recognition and Classification Based on Deep Voxel Convolution Neural Network[J]. Acta Optica Sinica, 2019,39(4):314-324.
[9]	RIEGLER G, ULUSOY A O, GEIGER A. Octnet: Learning Deep 3D Representations at High Resolutions[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6620-6629.
[10]	SU H, MAJI S, KALOGERAKIS E, et al. Multi-view Convolutional Neural Networks for 3D Shape Recognition[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 945-953.
[11]	FENG Y, ZHANG Z, ZHAO X, et al. GVCNN: Group-view Convolutional Neural Networks for 3D Shape Recognition[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 264-272.
[12]	KALOGERAKIS E, AVERKIOU M, MAJI S, et al. 3D Shape Segmentation with Projective Convolutional Networks[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6630-6639.
[13]	SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-scale Image Recognition[C]// Proceedings of the 2015 3rd International Conference on Learning Representations. San Diego: International Conference on Learning Representations, 2015: 1409-1423.
[14]	HUANG H, KALOGERAKIS E, CHAUDHURI S, et al. Learning Local Shape Descriptors from Part Correspondences with Multi View Convolutional Networks[J]. ACM Transactions on Graphics, 2018,37(1):6-20.
[15]	QI C R, SU H, MO K, et al. Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 77-85.
[16]	QI C R, YI L, SU H, et al. Pointnet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space[C]// Proceedings of the 2017 International Conference on Neural Information Processing Systems. Vancouver: Neural Information Processing Systems Foundation, 2017: 5100-5109.
[17]	WANG Y, SUN Y, LIU Z, et al. Dynamic Graph CNN for Learning on Point Clouds[J]. ACM Transactions on Graphics, 2019,38(5):146-160.
[18]	KLOKOV R, LEMPITSKY V. Escape from Cells: Deep Kd-networks for the Recognition of 3D Point Cloud Models[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 863-872.
[19]	LI Y, BU R, SUN M, et al. Pointcnn: Convolution on X-transformed Points[C]// Proceedings of the 2018 International Conference on Neural Information Processing Systems. Vancouver: Neural Information Processing Systems Foundation, 2018: 820-830.
[20]	ZHANG X, ZHOU X, LIN M, et al. Shufflenet: an Extremely Efficient Convolutional Neural Network for Mobile Devices[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 6848-6856.
[21]	CHEN C, FRAGONARA L Z, TSOURDOS A. Go Wider: An Efficient Neural Network for Point Cloud Analysis via Group Convolutions[J/OL]. [2019-09-23]. https: //arxiv.org/pdf/1909.10431.pdf.
[22]	LANDOLA F, MOSKEWICZ M, KARAYEV S, et al. Densenet: Implementing Efficient Convnet Descriptor Pyramids[J/OL]. [2014-04-07]. https: //arxiv.org/pdf/1404.1869.pdf.
[23]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 770-778.
[24]	YI L, KIM V, CEYLAN D, et al. A Scalable Active Framework for Region Annotation in 3D Shape Collections[J]. ACM Transactions on Graphics, 2016,35(6):1-12.
[25]	ARMENI I, SENER O, ZAMIR A R, et al. 3D Semantic Parsing of Large-scale Indoor Spaces[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 1534-1543.
[26]	ENGELMANN F, KONTOGIANNI T, HERMANS A, et al. Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops. Piscataway: IEEE, 2017: 716-724.

模型	Data Presentation	ModelNet 40 准确率 / %	ModelNet 10 准确率 /%
3D ShapeNet^[5]	Volumetric	77.3	83.5
VoxNet^[6] MVCNN^[10]	Volumetric 12 Views	83.0 89.9	92.0
PointNet^[15] PointNet++^[16] Kd-Net^[18] DGCNN^[17] Ours	点云点云点云点云点云	89.2 90.7 91.8 92.2 92.5	93.6

模型	准确率 / %
PointNet^[15] SPRNet(Graph Convolution+Dense Connection) Geometric Feature Branch	89.2 90.0 92.3
MFFCNN (Late Fusion) MFFCNN (Attention Fusion)	92.4 92.5

算法	S3DIS		vKITTI		ShapeNet Parts
算法	mIoU	OA	mIoU	OA	mIoU
PointNet^[15]	47.6	78.5	34.4	79.7	83.7
MS+CU^[26] G+RCU^[26] PointNet++^[16] DGCNN^[17] Ours	47.8 49.7 56.1 57.2	79.2 81.1 84.1 84.9	36.2 38.5	80.6 82.3	85.1 85.1 85.5