压缩深层神经网络隐藏层维度对其分类性能的影响

doi:10.16180/j.cnki.issn1007-7820.2019.01.0015

Abstract

Abstract:

In order to make deep neural network get better generalization property and reduce training time. Inspired by compressed neural network, compressing dimension of the hidden layers according to different compression ratio, and based on the traditional compressed deep neural network, adding classification layer to the top layer, then deep neural network gets the ability of classification. Experiments performed on MNIST handwritten dataset show that using properly compress deep neural network instead of uncompressed network would get better classification results, and saving lots of training time.

Key words: deep neural network, classification, compression ratio, generalization property, training time, deep learning

CLC Number:

TP183

CHENG Lingfei,HE Yang,ZHANG Peiling,LI Yan. Classification Performance of Compressing Dimensionality of Hidden Layer of Deep Neural Network[J].Electronic Science and Technology, 2019, 32(1): 72-75.

Figures/Tables 5

Figure 1.

Figure 2.

Table 1

Table 2

Figure 3.

References 16

[1]	Hinton G E, Osindero S, Teh Y W . A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006,18(7):1711-1800. doi: 10.1162/neco.2006.18.7.1711 pmid: 16764519
[2]	Ranzato M, Poultney C, Chopra S, et al. Efficient learning of sparse representations with an energy-based model [C]. Vancouver:Neural Information Processing System, 2006.
[3]	Bengio Y, Lamblin P, Popovici D. Greedy layer-wise training of deep networks [C].Vancouver:Neural Information Processing System, 2006.
[4]	Sicuranza G L, Ramponi G, Marsi S . Artificial neural network for image compression[J]. Electronic Letters, 2002,26(7):477-479. doi: 10.1049/el:19900310
[5]	Carrato S, Marsi S . Parallel structure based on neural networks for image compression[J]. Electronic Letters, 2002,28(12):1152-1153. doi: 10.1049/el:19920727
[6]	Qiu G, Varley M R, Terrell T J . Image compression by edge pattern learning using multilayer perceptions[J]. Electronic Letters, 1993(29):601-603. doi: 10.1049/el:19930403
[7]	Namphol A, Chin S H, Arozullah M . Image compression with a hierarchical neural networks[J]. IEEE Transactions on Aerospace and Electronic Systems, 1996,32(1):326-338. doi: 10.1109/7.481272
[8]	Hussain F, Jeong J . Efficient deep neural networks for digit image compression employing rectified linear neurons[J]. Journal of Sensors, 2016(15):1-7.
[9]	Hinton G E, Salakhutdinow R R . Reducing the dimensionality of the data with neural network[J]. Science, 2006,313(5786):504-507. doi: 10.1126/science.1127647 pmid: 16873662
[10]	Hinton G E . A practical guide to training restricted boltzmann machines[J]. Momentun, 2012,9(1):599-619. doi: 10.1007/978-3-642-35289-8_32
[11]	Mohamed A, Hinton G E, Penn G. Understanding how deep belief networks perform acoustic modeling [C]. New York:Acoustics, Speech and Signal Processing, 2012.
[12]	Seltzer M L, Yu D, Wang Y Q. An investigation of deep neural networks for noise robust speech recognition [C]. New York:Acoustics, Speech and Signal Processing, 2013.
[13]	薛皓天, 杨晶东, 谈凯德 . 一种改进的BP神经网络在手写体识别上的应用[J]. 电子科技, 2015,28(5):20-23. doi: 10.16180/j.cnki.issn1007-7820.2015.05.006
	Xue Haotian, Yang Jingdong, Tan Kaide . Application of an improved BP neural network in handwriting recognition[J]. Electronic Science and Technology, 2015,28(5):20-23. doi: 10.16180/j.cnki.issn1007-7820.2015.05.006
[14]	Hinton G E . Traning products of experts by minimizing contrastive divergence[J]. Neural Computation, 2002,14(8):1711-1800. doi: 10.1162/089976602760128018
[15]	Rumelhart D E, Hinton G E, Williams R J . Learning representations by back-propagating errors[J]. Nature, 1988,323(6088):533-536. doi: 10.1016/B978-1-4832-1446-7.50035-2
[16]	Lee Q V, Ngiam J, Coates A, et al. On optimization methods for deep learning [C].Washington: Proceeding of the 28 ^th International Conference on Machine Learning , 2011.

网络层	类型	神经元数目	激活函数
1	输入层	784	有
2	全连接		有
3	全连接	784/392/196/98	有
4	全连接		无
5	分类层	10	无

压缩比例	分类误差/%	训练时间/h
1:1	11.602 3	18.2
2:1	13.334 1	4.6
4:1	11.425 5	4.5
8:1	30.950 6	3.7

Classification Performance of Compressing Dimensionality of Hidden Layer of Deep Neural Network

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 16

Related Articles 15

Metrics

Comments

Recommended 0

[1]	CHU Ping,NI Wei. Design of FPGA-Based SqueezeNet Inference Accelerator [J]. Electronic Science and Technology, 2022, 35(2): 20-26.
[2]	LI Hui,WANG Yicheng. CNNCIFG-Attention Model for Text Sentiment Classifcation [J]. Electronic Science and Technology, 2022, 35(2): 46-51.
[3]	LI Na,GAO Bo,XIE Zongfu. Research on Scheduling Method of Layered Heterogeneous Signal Processing Platform [J]. Electronic Science and Technology, 2022, 35(2): 7-13.
[4]	CHENG Xiaoya,ZHANG Lei. Research on Occluded Face Recognition Method Based on Deep Learning [J]. Electronic Science and Technology, 2022, 35(1): 35-39.
[5]	ZHAO Chong,CHI Mengmeng,CHU Cong,ZHANG Peng. Research on Motion Simulation and Visual Recognition Algorithm of Guide Dog Walking Mechanism [J]. Electronic Science and Technology, 2021, 34(9): 66-72.
[6]	MA Lixin,DOU Chenfei,SONG Chencan,YANG Tianxiao. Insulator Nondestructive Testing Based on Feature Fusion CNN [J]. Electronic Science and Technology, 2021, 34(7): 26-30.
[7]	HAO Miao,CHEN Linqiang. Chinese Microblog Polarity Classification Based on Hownet and PMI [J]. Electronic Science and Technology, 2021, 34(7): 50-55.
[8]	ZHANG Pengfei,WEI Cunwei,LIU Xiankang,LIU Anran,YANG Ou,LIN Yonglin. Research on Ship Target Recognition Method of Narrowband Radar [J]. Electronic Science and Technology, 2021, 34(6): 50-55.
[9]	LIU Shu,SHAO Jie,ZHANG Yiting,ZHANG Shanzhang. ECG Classification Based on Bispectrum and Spectral Features [J]. Electronic Science and Technology, 2021, 34(5): 42-46.
[10]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.
[11]	SHAO Hang,WANG Yongxiong,QIN Yulong. Digital Image Composition Optimization Based on Salient Feature Algorithm [J]. Electronic Science and Technology, 2021, 34(3): 36-42.
[12]	HE Zhonglin,ZHOU Ping. Research on Sorting of Retired Lithium Battery Based on Radial Basis Function Neural Network [J]. Electronic Science and Technology, 2021, 34(2): 38-44.
[13]	ZHU Bin,LIU Zilong. Image Classification Method Using Convolutional Neural Network Based on New Initial Module [J]. Electronic Science and Technology, 2021, 34(2): 52-56.
[14]	ZHOU Xiaobo,ZOU Renling,LU Xuhua,WANG Haibin,ZHANG Junxiang. Research Progress of Surface Electromyography Signal Classifier Based on Artificial Neural Network [J]. Electronic Science and Technology, 2021, 34(2): 62-67.
[15]	YAN Chao,SUN Zhanquan,TIAN Engang,ZHAO Yangyang,FAN Xiaoyan. Research Progress of Medical Image Segmentation Based on Deep Learning [J]. Electronic Science and Technology, 2021, 34(2): 7-11.