多特征融合的三维模型识别与分割

doi:10.19665/j.issn1001-2400.2020.04.020

Abstract

Abstract:

Current methods focusing on 3D model recognition and segmentation have to some extent ignored the relationship between the high-level global single-point features and the low-level local geometric features of those models, resulting in poor recognition results. A multi-feature fusion approach which takes into consideration the aforementioned ignored relationship is proposed. First, a global single-point network is established to extract the global single-point features with high-level semantic recognition ability by increasing both the width of convolution kernel and the depth of the network. Second, an attentional fusion layer is constructed to learn the implicit relationship between global single-point features and local geometric features to fully explore the fine-grained geometric features that can better represent model categories. Finally, the global single-point features and fine-grained geometric features are further fused to achieve the complementation of advantages and enhance the feature richness. Experimental verification is carried out on the 3D model recognition datasets ModelNet40, ModelNet10 and segmentation datasets ShapeNet Parts, S3DIS, vKITTI, respectively, and comparison with current mainstream recognition algorithms shows that the proposed algorithm not only has higher recognition and segmentation accuracy, but also has stronger robustness.

Key words: 3D point cloud, object recognition, semantic segmentation, attention fusion, deep learning, convolutional neural networks

CLC Number:

TP391

DANG Jisheng,YANG Jun. 3D model recognition and segmentation based on multi-feature fusion[J].Journal of Xidian University, 2020, 47(4): 149-157.

Figures/Tables 10

References 26

[1]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet Classification with Deep Convolutional Neural Networks[C]// Proceedings of the 2012 25th International Conference on Neural Information Processing Systems: 1. Vancouver: Neural Information Processing Systems Foundation, 2012: 1097-1105.
[2]	张金刚, 方圆, 袁豪, 等. 一种识别表情序列的卷积神经网络[J]. 西安电子科技大学学报, 2018,45(1):150-155.
	ZHANG Jingang, FANG Yuan, YUAN Hao, et al. Multiple Convolutional Neural Networks for Facial Expression Sequence Recognition[J]. Journal of Xidian University, 2018,45(1):150-155.
[3]	朱苏雅, 杜建超, 李云松, 等. 采用U-Net 卷积网络的桥梁裂缝检测方法[J]. 西安电子科技大学学报, 2019,6(4 ): 35-42.
	ZHU Suya, Du Jianchao, LI Yunsong, et al. Method for Bridge Crack Detection Based on the U-Net Convolutional Network[J]. Journal of Xidian University, 2019,6(4):35-42.
[4]	刘道华, 崔玉爽, 赵岩松, 等. 一种改进卷积神经网络的教学图像检索方法[J]. 西安电子科技大学学报, 2019(3):52-58.
	LIU Daohua, CUI Yushuang, ZHAO Yansong, et al. Method for Retrieving the Teaching Image Based on the Improved Convolutional Neural Networks[J]. Journal of Xidian University, 2019(3):52-58.
[5]	WU Z, SONG S, KHOSLA A, et al. 3D Shapenets: a Deep Representation for Volumetric Shapes[C]// Proceedings of the 2015 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2015: 1912-1920.
[6]	MATURANA D, SCHERER S. Voxnet: A 3D Convolutional Neural Network for Real-time Object Recognition[C]// Proceedings of the 2015 IEEE International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2015: 922-928.
[7]	杨军, 王亦民. 基于深度卷积神经网络的三维模型识别[J]. 重庆邮电大学学报, 2019,31(2):253-260.
	YANG Jun, WANG Yimin. 3D Model Recognition and Classification Based on Deep Convolution Neural Network[J]. Journal of Chongqing University of Posts and Telecommunications, 2019,31(2):253-260.
[8]	杨军, 王顺, 周鹏. 基于深度体素卷积神经网络的三维模型识别分类[J]. 光学学报, 2019,39(4):314-324.
	YANG Jun, WANG Shun, ZHOU Peng. 3D Model Recognition and Classification Based on Deep Voxel Convolution Neural Network[J]. Acta Optica Sinica, 2019,39(4):314-324.
[9]	RIEGLER G, ULUSOY A O, GEIGER A. Octnet: Learning Deep 3D Representations at High Resolutions[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6620-6629.
[10]	SU H, MAJI S, KALOGERAKIS E, et al. Multi-view Convolutional Neural Networks for 3D Shape Recognition[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 945-953.
[11]	FENG Y, ZHANG Z, ZHAO X, et al. GVCNN: Group-view Convolutional Neural Networks for 3D Shape Recognition[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 264-272.
[12]	KALOGERAKIS E, AVERKIOU M, MAJI S, et al. 3D Shape Segmentation with Projective Convolutional Networks[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6630-6639.
[13]	SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-scale Image Recognition[C]// Proceedings of the 2015 3rd International Conference on Learning Representations. San Diego: International Conference on Learning Representations, 2015: 1409-1423.
[14]	HUANG H, KALOGERAKIS E, CHAUDHURI S, et al. Learning Local Shape Descriptors from Part Correspondences with Multi View Convolutional Networks[J]. ACM Transactions on Graphics, 2018,37(1):6-20.
[15]	QI C R, SU H, MO K, et al. Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 77-85.
[16]	QI C R, YI L, SU H, et al. Pointnet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space[C]// Proceedings of the 2017 International Conference on Neural Information Processing Systems. Vancouver: Neural Information Processing Systems Foundation, 2017: 5100-5109.
[17]	WANG Y, SUN Y, LIU Z, et al. Dynamic Graph CNN for Learning on Point Clouds[J]. ACM Transactions on Graphics, 2019,38(5):146-160.
[18]	KLOKOV R, LEMPITSKY V. Escape from Cells: Deep Kd-networks for the Recognition of 3D Point Cloud Models[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 863-872.
[19]	LI Y, BU R, SUN M, et al. Pointcnn: Convolution on X-transformed Points[C]// Proceedings of the 2018 International Conference on Neural Information Processing Systems. Vancouver: Neural Information Processing Systems Foundation, 2018: 820-830.
[20]	ZHANG X, ZHOU X, LIN M, et al. Shufflenet: an Extremely Efficient Convolutional Neural Network for Mobile Devices[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 6848-6856.
[21]	CHEN C, FRAGONARA L Z, TSOURDOS A. Go Wider: An Efficient Neural Network for Point Cloud Analysis via Group Convolutions[J/OL]. [2019-09-23]. https: //arxiv.org/pdf/1909.10431.pdf.
[22]	LANDOLA F, MOSKEWICZ M, KARAYEV S, et al. Densenet: Implementing Efficient Convnet Descriptor Pyramids[J/OL]. [2014-04-07]. https: //arxiv.org/pdf/1404.1869.pdf.
[23]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 770-778.
[24]	YI L, KIM V, CEYLAN D, et al. A Scalable Active Framework for Region Annotation in 3D Shape Collections[J]. ACM Transactions on Graphics, 2016,35(6):1-12.
[25]	ARMENI I, SENER O, ZAMIR A R, et al. 3D Semantic Parsing of Large-scale Indoor Spaces[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 1534-1543.
[26]	ENGELMANN F, KONTOGIANNI T, HERMANS A, et al. Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops. Piscataway: IEEE, 2017: 716-724.

模型	Data Presentation	ModelNet 40 准确率 / %	ModelNet 10 准确率 /%
3D ShapeNet^[5]	Volumetric	77.3	83.5
VoxNet^[6] MVCNN^[10]	Volumetric 12 Views	83.0 89.9	92.0
PointNet^[15] PointNet++^[16] Kd-Net^[18] DGCNN^[17] Ours	点云点云点云点云点云	89.2 90.7 91.8 92.2 92.5	93.6

模型	准确率 / %
PointNet^[15] SPRNet(Graph Convolution+Dense Connection) Geometric Feature Branch	89.2 90.0 92.3
MFFCNN (Late Fusion) MFFCNN (Attention Fusion)	92.4 92.5

算法	S3DIS		vKITTI		ShapeNet Parts
算法	mIoU	OA	mIoU	OA	mIoU
PointNet^[15]	47.6	78.5	34.4	79.7	83.7
MS+CU^[26] G+RCU^[26] PointNet++^[16] DGCNN^[17] Ours	47.8 49.7 56.1 57.2	79.2 81.1 84.1 84.9	36.2 38.5	80.6 82.3	85.1 85.1 85.5

3D model recognition and segmentation based on multi-feature fusion

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 26

Related Articles 15

Metrics

Comments

Recommended 10

[1]	LIU Jiawei,ZHANG Wenhui,KOU Xiaoli,LI Yanni. Harnessing adversarial examples via input denoising and hidden information restoring [J]. Journal of Xidian University, 2021, 48(6): 23-31.
[2]	LI Peng,FENG Cunqian,XU Xuguang,TANG Zixiang. Ballistic target fretting classification network based on Bayesian optimization [J]. Journal of Xidian University, 2021, 48(5): 139-148.
[3]	YAN Jia,CAO Yudong,REN Jiaxing,CHEN Donghao,LI Xiaohui. Deep asymmetric compression Hashing algorithm [J]. Journal of Xidian University, 2021, 48(5): 212-221.
[4]	NING Yang,DU Jianchao,HAN Shuo,YANG Chuankai. Fire segmentation based on the improved DeeplabV3+ and the analytical method for fire development [J]. Journal of Xidian University, 2021, 48(5): 38-46.
[5]	ZHOU Peng,YANG Jun. Semantic segmentation of remote sensing images based on neural architecture search [J]. Journal of Xidian University, 2021, 48(5): 47-57.
[6]	QI Yanjun,KONG Yueping,WANG Jiajing,ZHU Xudong. Gait recognition method combining LSTM and CNN [J]. Journal of Xidian University, 2021, 48(5): 78-85.
[7]	SONG Jianfeng,MIAO Qiguang,WANG Chongxiao,XU Hao,YANG Jin. Multi-scale single object tracking based on the attention mechanism [J]. Journal of Xidian University, 2021, 48(5): 110-116.
[8]	ZHANG Yuhao,CHENG Peitao,ZHANG Shuhao,WANG Xiumei. Lightweight image super-resolution with the adaptive weight learning network [J]. Journal of Xidian University, 2021, 48(5): 15-22.
[9]	HUI Haisheng,ZHANG Xueying,WU Zelin,LI Fenglian. Method for stroke lesion segmentation using the primary-auxiliary path attention compensation network [J]. Journal of Xidian University, 2021, 48(4): 200-208.
[10]	SUN Haojie,LI Miaoyu,ZHANG Panpan,XU Pengfei. Self-supervised facial asymmetry learning for automatic evaluation of facial paralysis [J]. Journal of Xidian University, 2021, 48(3): 115-122.
[11]	GUO Zekun,TIAN Long,HAN Ning,WANG Penghui,LIU Hongwei,CHEN Bo. Radar HRRP based few-shot target recognition with CNN-SSD [J]. Journal of Xidian University, 2021, 48(2): 7-14.
[12]	ZHANG Hua,GAO Haoran,YANG Xingguo,LI Wenmin,GAO Fei,WEN Qiaoyan. TargetedFool:an algorithm for achieving targeted attacks [J]. Journal of Xidian University, 2021, 48(1): 149-159.
[13]	YANG Hongyu,ZENG Renyun. Method for assessment of network security situation with deep learning [J]. Journal of Xidian University, 2021, 48(1): 183-190.
[14]	ZHANG Lu,SUN Rong,LIU Jingwei. Cloned piggybacking framework for distributed storage [J]. Journal of Xidian University, 2020, 47(6): 139-147.
[15]	HU Jianwei,ZHAO Wei,CUI Yanpeng,CUI Junjie. PHP code vulnerability mining technology based on theimproved ASTNN [J]. Journal of Xidian University, 2020, 47(6): 164-173.