压缩深层神经网络隐藏层维度对其分类性能的影响

doi:10.16180/j.cnki.issn1007-7820.2019.01.0015

电子科技 ›› 2019, Vol. 32 ›› Issue (1): 72-75.doi: 10.16180/j.cnki.issn1007-7820.2019.01.0015

压缩深层神经网络隐藏层维度对其分类性能的影响

成凌飞¹,贺扬²,张培玲¹,李艳²

^1. 河南理工大学物理与电子信息学院,河南焦作 454000
^2. 河南理工大学电气工程与自动化学院,河南焦作 454000

收稿日期:2017-12-19 出版日期:2019-01-15 发布日期:2018-12-29
作者简介:成凌飞(1971-),男,博士,教授。研究方向:矿井通信及监控。|贺扬(1990-),男,硕士研究生。研究方向:通信技术与信号处理。
基金资助:
国家自然科学基金(61501175);河南省教育厅科学技术研究重点项目(15A510008);河南理工大学博士基金(B2015-33)

Classification Performance of Compressing Dimensionality of Hidden Layer of Deep Neural Network

CHENG Lingfei¹,HE Yang²,ZHANG Peiling¹,LI Yan²

^1. School of Physics and Electronic Information,Henan Polytechnic University,Jiaozuo 454000,China
^2. School of Electrical Engineering and Automation,Henan Polytechnic University,Jiaozuo 454000,China

Received:2017-12-19 Online:2019-01-15 Published:2018-12-29
Supported by:
National Natural Science Foundation of China(61501175);Key Project of Science and Technology Research of Henan Educational Committee(15A510008);Doctoral Fund of Henan Polytechnic University(B2015-33)

摘要/Abstract

摘要：

为了使深层神经网络具有更好的泛化能力、少量训练时间的性能。文中汲取压缩神经网络的思想,通过将深层神经网络的输入层与隐藏层按照不同的比例将隐藏层的维度进行压缩,并在传统压缩深层神经网络的基础上,在其顶层添加一个分类层,使深层神经网络拥有分类的能力。实验将构建的深层神经网络应用于MNIST手写数据集的分类测试,结果表明,经过适当压缩的深层神经网络比未被压缩的深层神经网络具有更好的分类效果,而且节省了大量的训练时间。

关键词: 深层神经网络, 分类, 压缩比例, 泛化能力, 训练时间, 深度学习

Abstract:

In order to make deep neural network get better generalization property and reduce training time. Inspired by compressed neural network, compressing dimension of the hidden layers according to different compression ratio, and based on the traditional compressed deep neural network, adding classification layer to the top layer, then deep neural network gets the ability of classification. Experiments performed on MNIST handwritten dataset show that using properly compress deep neural network instead of uncompressed network would get better classification results, and saving lots of training time.

Key words: deep neural network, classification, compression ratio, generalization property, training time, deep learning

中图分类号:

TP183

成凌飞,贺扬,张培玲,李艳. 压缩深层神经网络隐藏层维度对其分类性能的影响[J]. 电子科技, 2019, 32(1): 72-75.

CHENG Lingfei,HE Yang,ZHANG Peiling,LI Yan. Classification Performance of Compressing Dimensionality of Hidden Layer of Deep Neural Network[J]. Electronic Science and Technology, 2019, 32(1): 72-75.

图/表 5

图1

图2

表1

表2

图3

参考文献 16

[1]	Hinton G E, Osindero S, Teh Y W . A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006,18(7):1711-1800. doi: 10.1162/neco.2006.18.7.1711 pmid: 16764519
[2]	Ranzato M, Poultney C, Chopra S, et al. Efficient learning of sparse representations with an energy-based model [C]. Vancouver:Neural Information Processing System, 2006.
[3]	Bengio Y, Lamblin P, Popovici D. Greedy layer-wise training of deep networks [C].Vancouver:Neural Information Processing System, 2006.
[4]	Sicuranza G L, Ramponi G, Marsi S . Artificial neural network for image compression[J]. Electronic Letters, 2002,26(7):477-479. doi: 10.1049/el:19900310
[5]	Carrato S, Marsi S . Parallel structure based on neural networks for image compression[J]. Electronic Letters, 2002,28(12):1152-1153. doi: 10.1049/el:19920727
[6]	Qiu G, Varley M R, Terrell T J . Image compression by edge pattern learning using multilayer perceptions[J]. Electronic Letters, 1993(29):601-603. doi: 10.1049/el:19930403
[7]	Namphol A, Chin S H, Arozullah M . Image compression with a hierarchical neural networks[J]. IEEE Transactions on Aerospace and Electronic Systems, 1996,32(1):326-338. doi: 10.1109/7.481272
[8]	Hussain F, Jeong J . Efficient deep neural networks for digit image compression employing rectified linear neurons[J]. Journal of Sensors, 2016(15):1-7.
[9]	Hinton G E, Salakhutdinow R R . Reducing the dimensionality of the data with neural network[J]. Science, 2006,313(5786):504-507. doi: 10.1126/science.1127647 pmid: 16873662
[10]	Hinton G E . A practical guide to training restricted boltzmann machines[J]. Momentun, 2012,9(1):599-619. doi: 10.1007/978-3-642-35289-8_32
[11]	Mohamed A, Hinton G E, Penn G. Understanding how deep belief networks perform acoustic modeling [C]. New York:Acoustics, Speech and Signal Processing, 2012.
[12]	Seltzer M L, Yu D, Wang Y Q. An investigation of deep neural networks for noise robust speech recognition [C]. New York:Acoustics, Speech and Signal Processing, 2013.
[13]	薛皓天, 杨晶东, 谈凯德 . 一种改进的BP神经网络在手写体识别上的应用[J]. 电子科技, 2015,28(5):20-23. doi: 10.16180/j.cnki.issn1007-7820.2015.05.006
	Xue Haotian, Yang Jingdong, Tan Kaide . Application of an improved BP neural network in handwriting recognition[J]. Electronic Science and Technology, 2015,28(5):20-23. doi: 10.16180/j.cnki.issn1007-7820.2015.05.006
[14]	Hinton G E . Traning products of experts by minimizing contrastive divergence[J]. Neural Computation, 2002,14(8):1711-1800. doi: 10.1162/089976602760128018
[15]	Rumelhart D E, Hinton G E, Williams R J . Learning representations by back-propagating errors[J]. Nature, 1988,323(6088):533-536. doi: 10.1016/B978-1-4832-1446-7.50035-2
[16]	Lee Q V, Ngiam J, Coates A, et al. On optimization methods for deep learning [C].Washington: Proceeding of the 28 ^th International Conference on Machine Learning , 2011.

网络层	类型	神经元数目	激活函数
1	输入层	784	有
2	全连接		有
3	全连接	784/392/196/98	有
4	全连接		无
5	分类层	10	无

压缩比例	分类误差/%	训练时间/h
1:1	11.602 3	18.2
2:1	13.334 1	4.6
4:1	11.425 5	4.5
8:1	30.950 6	3.7

压缩深层神经网络隐藏层维度对其分类性能的影响

Classification Performance of Compressing Dimensionality of Hidden Layer of Deep Neural Network

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 16

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	李辉,王一丞. 基于CNNCIFG-Attention模型的文本情感分类[J]. 电子科技, 2022, 35(2): 46-51.
[2]	李娜,高博,谢宗甫. 分层异构信号处理平台调度方法研究[J]. 电子科技, 2022, 35(2): 7-13.
[3]	程晓雅,张雷. 基于深度学习的有遮挡人脸识别方法研究[J]. 电子科技, 2022, 35(1): 35-39.
[4]	赵崇,迟蒙蒙,储聪,张鹏. 导盲犬行走机构运动仿真及其视觉识别算法研究[J]. 电子科技, 2021, 34(9): 66-72.
[5]	马立新,豆晨飞,宋晨灿,杨天笑. 基于特征融合卷积神经网络的绝缘子无损检测[J]. 电子科技, 2021, 34(7): 26-30.
[6]	郝苗,陈临强. PMI与Hownet结合的中文微博情感分析[J]. 电子科技, 2021, 34(7): 50-55.
[7]	翟岳仙,刘翔,宋家琳. 基于迁移学习策略的肝纤维化分期诊断方法[J]. 电子科技, 2021, 34(6): 11-16.
[8]	张朋飞,魏存伟,刘先康,刘安然,杨欧,林永霖. 窄带舰船目标识别方法研究[J]. 电子科技, 2021, 34(6): 50-55.
[9]	刘姝,邵杰,张颐婷,张善章. 基于双谱和谱特征的心电信号分类方法[J]. 电子科技, 2021, 34(5): 42-46.
[10]	薛永杰,巨志勇. 基于改进AlexNet的鱼类识别算法[J]. 电子科技, 2021, 34(4): 12-17.
[11]	邵杭,王永雄,秦宇龙. 基于深度学习与显著性的数字图像构图优化[J]. 电子科技, 2021, 34(3): 36-42.
[12]	朱斌,刘子龙. 基于新型初始模块的卷积神经网络图像分类方法[J]. 电子科技, 2021, 34(2): 52-56.
[13]	周小波,邹任玲,卢旭华,王海滨,张俊翔. 基于人工神经网络的表面肌电信号分类器研究进展[J]. 电子科技, 2021, 34(2): 62-67.
[14]	闫超,孙占全,田恩刚,赵杨洋,范小燕. 基于深度学习的医学图像分割技术研究进展[J]. 电子科技, 2021, 34(2): 7-11.
[15]	白兴宇,华生辉,姜煜,张敏. 一种基于宽带谱处理的机械系统故障检测方法[J]. 电子科技, 2021, 34(12): 25-29.