一种用于点云分类的轻量级深度神经网络

doi:10.19665/j.issn1001-2400.2020.02.007

摘要/Abstract

摘要：

在使用深度神经网络进行点云分类时,可以将点云转化为三维体素或是从多个角度将网格模型渲染成图片后进行处理,但转换过程会耗费额外的计算和存储资源;而使用原始点云作为输入的方法,其网络规模和计算复杂度又给算法在嵌入式环境部署带来困难。在对这些算法深入研究的基础上,提出一种轻量级的双路径神经网络模型,无须转换点云数据格式,使用0.8兆浮点数的参数量,达到和主流方法相当的分类精度。双路径结构利用点云的点内和点间表示方式,在避免多尺度学习引入的复杂结构和计算的同时,挖掘了全局特征和局部细粒度特征。实验结果表明,该网络模型可对ModelNet40和MNIST数据集进行准确分类,且设计具有合理性。

关键词: 机器学习, 深度学习, 模式识别, 点云分类

Abstract:

For point cloud classification, deep learning based methods use operations like voxelization to generate regular 3D grids or render the 3D mesh into a collection of images from multiple angles. However, the conversion will introduce additional computing and storage consumption. Some methods directly consume the raw point cloud. But their network scale and computational complexity make it difficult for them to deploy in embedded environments. On the basis of intensive studies of these algorithms, a novel lightweight dual path way network is proposed in this paper. Without additional conversion, our network attains a comparable performance but has 0.8 million floating parameters only. With point-wise and neighbor-wise representations, our approach incorporates global and local features of the point cloud. Experimental results on ModelNet40 and MNIST data-set demonstrate that our method achieves a good accuracy, and prove the effectiveness of our design.

Key words: machine learning, deep learning, pattern recognition, point cloud classification

中图分类号:

TP391

闫林,刘凯,段玫妤. 一种用于点云分类的轻量级深度神经网络[J]. 西安电子科技大学学报, 2020, 47(2): 46-53.

YAN Lin,LIU Kai,DUAN Meiyu. Lightweight deep neural network for point cloud classification[J]. Journal of Xidian University, 2020, 47(2): 46-53.

图/表 15

图1

图2

图3

图4

图5

表1

表2

表3

表4

表5

表6

表7

图6

图7

图8

参考文献 21

[1]	张金刚, 方圆, 袁豪 , 等. 一种识别表情序列的卷积神经网络[J]. 西安电子科技大学学报, 2018,45(1):150-155.
[2]	ZHANG Jingang, FANG Yuan, YUAN Hao , et al. Multiple Convolutional Neural Networks for Facial Expression Sequence Recognition[J]. Journal of Xidian University, 2018,45(1):150-155.
[3]	朱苏雅, 杜建超, 李云松 , 等. 采用U-Net卷积网络的桥梁裂缝检测方法[J]. 西安电子科技大学学报, 2019,46(4):35-42.
[4]	ZHU Suya, Du Jianchao, LI Yunsong , et al. Method for Bridge Crack Detection Based on the U-Net Convolutional Networks[J]. Journal of Xidian University, 2019,46(4):35-42.
[5]	刘道华, 崔玉爽, 赵岩松 , 等. 一种改进卷积神经网络的教学图像检索方法[J]. 西安电子科技大学学报, 2019,46(3):52-58.
[6]	LIU Daohua, CUI Yushuang, ZHAO Yansong , et al. Method for Retrieving the Teaching Image Based on the Improved Convolutional Neural Network s[J]. Journal of Xidian University, 2019,46(3):52-58.
[7]	MATURANA D, SCHERER S , et al. VoxNet: A 3D Convolutional Neural Network forReal-time Object Recognition[C]// Proceedings of the 2015 IEEE International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2015: 922-928.
[8]	WANG P S, LIU Y, GUO Y X , et al. O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis[J]. ACM Transactions on Graphics, 2017,36(4):72.
[9]	RIEGLERG, ULUSOY A O, GEIGER A , et al. OctNet: Learning Deep 3D Representations at High Resolutions[C]// Proceedings of the 2017 30th IEEE International Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6620-6629.
[10]	SU H, MAJI S, KALOGERAKIS E , et al. Multi-view Convolutional Neural Networks for 3D Shape Recognition[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 945-953.
[11]	QI C R, SU H, NIEBNER M , et al. Volumetric and Multi-view CNNs for Object Classification on 3D Data[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 5648-5656.
[12]	HONG R, HU Z, WANG R , et al. Multi-view Object Retrieval via Multi-scale Topic Models[J]. IEEE Transactions on Image Processing, 2016,25(12):5814-5827.
[13]	QI C R, SU H, MO K , et al. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 77-85.
[14]	QI C R, YI L, SU H , et al. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space[C]// Advances in Neural Information Processing Systems: 2017. Vancouver: Neural Information Processing Systems Foundation, 2017: 5100-5109.
[15]	ZAHEER M, KOTTUR S, RAVANBAKHSH S , et al. Deep Sets[C]// Advances in Neural Information Processing Systems: 2017. Vancouver: Neural Information Processing Systems Foundation, 2017: 3392-3402.
[16]	WU Z, SONG S, KHOSLA A , et al. 3D ShapeNets:a Deep Representation for Volumetric Shapes[C]// Proceedings of the 2015 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2015: 1912-1920.
[17]	HUA B S, TRANM K, YEUNG S K , et al. Pointwise Convolutional Neural Networks[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 984-993.
[18]	LI Q D, GRIFFITHS J G , et al. Least Squares Ellipsoid Specific Fitting[C]// Proceedings-Geometric Modeling and Processing 2004. Washington: IEEE Computer Society, 2004: 335-340.
[19]	RAVANBAKHSH S, SCHNEIDER J, POCZOS B , et al. Deep Learning with Sets and Point Clouds[C]// Workshop Track Proceedings of the 2019 5th International Conference on Learning Representations. San Diego: International Conference on Learning Representations, 2019: 149805.
[20]	LECUN Y, BOTTOU L, BENGIO Y , et al. Gradient-based Learning Applied to Document Recognition[J]. Proceedings of the IEEE, 1998,86(11):2278-2324.
[21]	LIN M, CHEN Q, YAN S C , et al. Network in Network[C]// Conference Track Proceedings of the 2014 2nd International Conference on Learning Representations. San Diego: International Conference on Learning Representations, 2014: 149797.

层名称	输入维度	输出维度
点内表示	(N, 3)	(N, 1, 25)
Conv2D, [1, 1]	(N, 1, 25)	(N, 1, 64)
Conv2D, [1, 1]	(N, 1, 64)	(N, 1, 128)

层名称	输入维度	输出维度
点间表示	(N, 3)	(N, K, 25)
Conv2D, [1, 1]	(N, K, 25)	(N, K, 128)
Max pooling	(N, K, 128)	(N, 1, 128)
Conv2D, [1, 1]	(N, 1, 128)	(N, 1, 128)

层名称	输入维度	输出维度
Concatenation	(N, 1, 3)、(N, 1, 128)、(N, 1, 128)	(N, 1, 259)
Conv2D, [1, 1]	(N, 1, 259)	(N, 1, 256)
Conv2D, [1, 1]	(N, 1, 256)	(N, 1, 384)
Conv2D, [1, 1]	(N, 1, 384)	(N, 1, 512)
Max pooling	(N, 1, 512)	(512)
FC, 512	(512)	(512)
FC, 256	(512)	(256)
FC, 40	(256)	(40)
Softmax	(40)	(40)

名称	类平均精度/%	精度/%	数据类型	数据维度	权重数量(百万)	GFLOP
MVCNN^[8]	91.4	90.1	图片	60×244²	180	233
PointNet^[10]	86.2	89.2	点云	1024×3	3.5	14.7
PointNet++^[11]		90.7	点云	1024×3	4.1	26.9
PointWiseConv^[14]	81.4	86.1	点云	2048×3	1.8	16.8
VoxNet^[4]	83	85.9	体素	30³	0.9
ShapeNets^[13]	77.3	84.7	体素	30³	11
文中方法	89.2	91.1	点云	1024×3	0.8	18.4

名称	错误率/%	输入数据类型	输入数据维度
LeNet^[17]	0.80	图片	28²
Network in Network^[18]	0.47	图片	28²
PointNet^[10]	0.78	点云	512×3
PointNet++^[11]	0.51	点云	512×3
文中方法	0.59	点云	1024×3