索引边缘几何卷积神经网络用于点云分类

doi:10.19665/j.issn1001-2400.2022.02.024

摘要/Abstract

摘要：

点云学习因其在计算机视觉、自动驾驶和机器人技术等领域的广泛应用而受到越来越多的关注。点云数据具有稀疏性、无序性、有限性等特点,给基于深度卷积神经网络的点云分类任务带来很大困难,通常采用多视图或将点云转换为体素后使用卷积神经网络进行处理,但转换过程会带来局部特征信息丢失、计算效率低等问题。将原始点云直接输入到分类网络还存在参数量过多、网络规模复杂等问题,实时性任务处理仍需进一步优化。为使点云处理网络轻量化,以适用于实时点云分类任务,提出索引边缘几何卷积神经网络模型。首先对网络结构和超参进行裁剪并压缩,实现模型轻量化;其次,使用k近邻算法在每个卷积层上确定新的局部区域,加入相邻点之间的向量方向,将不同层的输出特征映射并进行索引跳跃连接,使局部特征信息损失进一步降低。在ModelNet40数据集上,该方法的分类准确率约为92.78%,与DGCNN相比,提高了约0.58%。实验结果表明,所提出的模型具有分类准确率高、轻量化特点,可部署于小型嵌入式设备。

关键词: 模型分类, 卷积神经网络, 点云, 轻量化

Abstract:

Point cloud learning has attracted more and more attention due to its wide application in many scientific fields,such as computer vision,automatic driving,and robot technology.The sparseness,disorder and finiteness of point cloud data present great difficulties to the point cloud classification task based on deep convolutional neural networks.The convolutional neural network often uses operations like voxelization and multi-view.However,the conversion process causes problems of local feature information loss and low computational efficiency.At present,the existing deep learning method oriented point cloud classification which consumes the raw point cloud has the problem of many traning parameters and complex model structures,thus making it difficult to process the real-time task.In order to lightweight the network to realize real-time point cloud classification tasks,on the basis of intensive studies of these problems,an index edge geometric convolution neural network model is proposed in this paper.First,in order to achieve lightweight,the network structure and hyperparameters are trimmed and compressed,respectively.Second,to enhance the accuracy and efficiency of algorithms,a new local region is determined on each convolution layer by using the KNN(K-Nearest Neighbor) algorithm,with the vector direction between adjacent points added,and the output features of each layer are inherited by an index hop link,which further reduces the loss of local feature information.Experiments show that the proposed network can achieve a classification accuracy of 92.78% during 42 min on the ModelNet40.Compared with the DGCNN(Dynamic Graph Convolutional Neural Network),the classification accuracy is improved by 0.58%.With the advantages of high classification accuracy and lightweight,this network model can be deployed in small embedded devices.

Key words: model classification, convolutional neural network, point cloud, lightweight

中图分类号:

TP391

周鹏,杨军. 索引边缘几何卷积神经网络用于点云分类[J]. 西安电子科技大学学报, 2022, 49(2): 207-217.

ZHOU Peng,YANG Jun. Index edge geometric convolution neural network for point cloud classification[J]. Journal of Xidian University, 2022, 49(2): 207-217.

图/表 11

图1

图2

表1

图3

图4

图5

图6

图7

图8

表2

表3

参考文献 24

[1]	刘道华, 崔玉爽, 赵岩松, 等. 一种改进卷积神经网络的教学图像检索方法[J]. 西安电子科技大学学报, 2019, 46(3):52-58.
	LIU Daohua, CUI Yushuang, ZHAO Yansong, et al. Method for Retrieving the Teaching Image Based on the Improved Convolutional Neural Network[J]. Journal of Xidian University, 2019, 46(3):52-58.
[2]	杨军, 王顺, 周鹏. 基于深度体素卷积神经网络的三维模型识别分类[J]. 光学学报, 2019, 39(4):314-324.
	YANG Jun, WANG Shun,Zhou Peng.Recognition and Classification for Three-Dimensional Model Based on Deep Voxel Convolution Neural Network[J]. Acta Optica Sinica, 2019, 39(4):314-324.
[3]	QI C R, HAO S, MO K, et al. PointNet:Deep Learning on Point Sets for 3D Classification and Segmentation[C]// Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:77-85.
[4]	QI C R, YI L, SU H, et al. PointNet++:Deep Hierarchical Feature Learning on Point Sets in a Metric Space[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017:5105-5114.
[5]	葛道辉, 李洪升, 张亮, 等. 轻量级神经网络架构综述[J]. 软件学报, 2020, 31(9):2627-2653.
	GE Daohui, LI Hongsheng, ZHANG Liang, et al. Survey of Lightweight Neural Network[J]. Journal of Software, 2020, 31(9):2627-2653.
[6]	闫林, 刘凯, 段玫妤. 一种用于点云分类的轻量级深度神经网络[J]. 西安电子科技大学学报, 2020, 47(2):46-53.
	YAN Lin, LIU Kai, DUAN Meiyu.Lightweight Deep Neural Network for Point Cloud Classification[J]. Journal of Xidian University, 2020, 47(2):46-53.
[7]	白静, 司庆龙, 秦飞巍. 轻量级实时点云分类网络LightPointNet[J]. 计算机辅助设计与图形学学报, 2019, 31(4):612-621.
	BAI Jing, SI Qinglong, QIN Feiwei. Lightweight Real-Time Point Cloud Classification Network LightPointNet[J]. Journal of Computer-Aided Design & Computer Graphics, 2019, 31(4):612-621.
[8]	LI M, Hu Y, ZHAO N, et al. LPCCNet:A Lightweight Network for Point Cloud Classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(6):962-966. doi: 10.1109/LGRS.2018.2889472
[9]	ZHANG M, YOU H X, KADAM P, et al. PointHop:An Explainable Machine Learning Method for Point Cloud Classification[J]. IEEE Transactions on Multimedia, 2020, 22(7):1744-1755. doi: 10.1109/TMM.2019.2963592
[10]	KUO CC J, ZHANG M, LI S, et al. Interpretable Convolutional Neural Networks via Feedforward Design[J]. Journal of Visual Communication and Image Representation, 2019, 60:346-359. doi: 10.1016/j.jvcir.2019.03.010
[11]	ZHANG M, WANG Y F, KADAM P, et al. Pointhop++:A Lightweight Learning Model on Point Sets for 3D Classification (2020)[J/OL].[2020-05-23]. https://arxiv.org/abs/2002.03281.
[12]	WANG Y, SUN Y, LIU Z, et al. Dynamic Graph CNN for Learning on Point Clouds[J]. ACM Transactions on Graphics, 2019, 38(5):1-12.
[13]	LIU W, SUN J, LI W, et al. Deep Learning on Point Clouds and Its Application:A Survey[J]. Sensors, 2019, 19(19):4188. doi: 10.3390/s19194188
[14]	LAN S, YU R, YU G, et al. Modeling Local Geometric Structure of 3D Point Clouds Using Geo-CNN[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:998-1008.
[15]	SU H, MAJI S, KALOGERAKIS E, et al. Multi-View Convolutional Neural Networks for 3D Shape Recognition[C]// Proceedings of the IEEE International Conference on Computer Vision.Piscataway:IEEE, 2015:945-953.
[16]	JOHNS E, LEUTENEGGER S, DAVISON A J. Pairwise Decomposition of Image Sequences for Active Multi-View Recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:3813-3822.
[17]	WU Z, SONG S, KHOSLA A, et al. 3D ShapeNets:A Deep Representation for Volumetric Shapes[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2015:1912-1920.
[18]	BROCK A, LIM T, RITCHIE J M, et al. Generative and Discriminative Voxel Modeling with Convolutional Neural Networks (2016)[J/OL].[2016-08-16]. https://arxiv.org/abs/1608.04236.
[19]	MATURANA D, SCHERER S. Voxnet:A 3D Convolutional Neural Network for Real-Time Object Recognition[C]// IEEE/RSJ International Conference on Intelligent Robots and Systems.Piscataway:IEEE, 2015:922-928.
[20]	ZHI S F, LIU Y X, LI X, et al. Towards Real-Time 3D Object Recognition:A Lightweight Volumetric CNN Framework Using Multitask Learning[J]. Computers & Graphics, 2018, 71:199-207. doi: 10.1016/j.cag.2017.10.007
[21]	HEGDE V, ZADEH R. FusionNet:3D Object Classification Using Multiple Data Representations (2016)[J/OL].[2018-06-15]. https://arxiv.org/abs/1607.05695.
[22]	ZHANG K, HAO M, WANG J, et al. Linked Dynamic Graph CNN:Learning on Point Cloud via Linking Hierarchical Features (2019)[J/OL].[2019-08-06]. https://arxiv.org/abs/1904.10014.
[23]	BEN-SHABAT Y, LINDENBAUM M, FISCHER A. 3D Point Cloud Classification and Segmentation Using 3D Modified Fisher Vector Representation for Convolutional Neural Networks (2017)[J/OL], [2017-11-12]. https://arxiv.org/abs/1711.08241.
[24]	LIU X, HAN Z, LIU Y S, et al. Point2Sequence: Learning the Shape Representation of 3D Point Clouds with An Attention-Based Sequence to Sequence Network[C]// Proceedings of the AAAI Conference on Artificial Intelligence.Hawaii:AAAI, 2019:8778-8785.

组号	名次	R	T	S	K	M	P	准确率/%
G1	No.1	64	64	128	1 024	512	256	92.91
G1	No.2	64	16	16	1 024	256	256	92.74
G1	No.3	64			1 024	256	256	91.71
G2	No.4	128			128	256	256	90.99
G2	No.5	128			128	256		91.64
G3	No.6	16			1 024	256		91.52
G3	No.7	64			1 024	256		92.17
G3	No.8	256			1 024	256		92.29
G3	No.9	512			1 024	256		92.09
G4	No.10	128			64	256		90.99
G4	No.11	128			256	256		91.92
G4	No.12	128			512	256		92.13
G4	No.13	128			1 024	256		92.78
G4	No.14	128			2 048	256		92.69

层	卷积核	步长	输出维度	参数个数
C1	1×3	1×1	64×n×1	256
C2	1×1	1×1	128×n×1	8 320
C3	1×1	1×1	1 024×n×1	132 096
Pooling	n×1	1×1	1 024×1	1 024
FC			256	262 400
输出			k	257k

输入	算法	params/M	ModelNet40	ModelNet10
Multi-view	MVCNN^[15]	~138.00	90.10
Multi-view	Pairwise^[16]	~138.00	90.70	92.80
Volumetric	3DShapeNets^[17]	~38.00	77.32	83.54
Volumetric	VRN Ensemble^[18]	~90.00	95.54	97.14
Volumetric	VRN^[18]	~18.00	91.33	93.61
Volumetric	Voxception^[18]		90.56	93.28
Volumetric	VoxNet^[19]	~0.92	83.00	92.00
Volumetric	LightNet^[20]	~0.30	86.90	93.39
Vol.+Mul.	FusionNet^[21]	~118.00	90.80	93.11
PointCloud	PointNet^[3]	3.48	89.20	93.08
PointCloud	PointNet(vanilla)^[3]	0.80	87.20	91.96
PointCloud	PointNet++^[4]	1.48	90.70
PointCloud	DGCNN^[12]	1.84	92.20
PointCloud	LDGCNN^[22]		92.90
PointCloud	3DmFV-Net^[23]	4.60	91.60	95.20
PointCloud	Point2Sequences^[24]		92.60	95.30
文中方法	IEGCNN	0.61	92.78	94.20