基于神经网络的文本标题生成原型系统设计

doi:10.16180/j.cnki.issn1007-7820.2021.05.007

Abstract

Abstract:

In view of the traditional manual methods cost a lot of manpower and time and can not deal with the problem of massive of non-standard texts, a prototype system of generating text titles is designed in the proposed study. In the prototype system, the non-standard text is calculated by the encoder-decoder model which is based on neural network to produce an accurate title. In the encoder part, the bidirectional long short-term memory neural network is adopted to make full use of the semantic connection between contexts. In the decoder part, one-way neural network is used for decoding operation, and attention mechanism is added to alleviate information loss and improve the effect of title generation. The evaluation indexes of ROUGE-1 and ROUGE-L obtained by experiments on LCSTS data set are 29.91 and 24.68, proving the effectiveness of the title generation prototype system.

Key words: artificial intelligence, natural language processing, neural network, title generation, prototype system, word vector, attention mechanism, generative technology

CLC Number:

TP391

ZHANG Shisen,SUN Xiankun,YIN Ling,LI Shixi. Design of Text Title Generation Prototype System Based on Neural Network[J].Electronic Science and Technology, 2021, 34(5): 35-41.

Figures/Tables 12

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Table 1

Figure 5.

Figure 6.

Table 2

Figure 7.

Figure 8.

Figure 9.

Table 3

References 20

[1]	Aone C, Okurowski M E, Gorlinsky J, et al. A trainable summarizer with knowledge acquired from robust NLP techniques[C]. Montreal:The Thirty-sixth Annual Meeting of the Association for Computational Linguistics, 1999.
[2]	Lin C Y. Training a selection function for extraction[C]. New York:The Eighth International Conference on Information and Knowledge Management, 1999.
[3]	Mihalcea R, Tarau P. TextRank:bringing order into texts[C]. Barcelona:Conference on Empirical Methods in Natural Language Processing, 2004.
[4]	Erkan, Radev. Dragomir R.LexRank: graph-based lexical centrality as salience in text summarization[J]. Journal of Artificial Intelligence Research, 2004,22(1):457-479. doi: 10.1613/jair.1523
[5]	Lopyrev K. Generating news headlines with recurrent neural networks[J]. Computer Science, 2015,2(2):159-165.
[6]	黄鑫池. 神经网络自动生成汉语新闻标题的应用[J]. 电子技术与软件工程, 2018(22):134-135.
	Huang Xinchi. Application of neural network to automatically generate Chinese news headlines[J]. Electronic Technology & Software Engineering, 2018(22):134-135.
[7]	Hou L W, Hu P, Chao B. Abstractive document summarization via neural model with joint attention[C]. Dalian:Proceedings of the Sixth Conference on Natural Language Processing and Chinese Computing, 2017.
[8]	Ranzato M A, Chopra S, Auli M, et al. Sequence level training with recurrent neural networks[J/OL]. (2016-05-06)[2019-11-28] .
[9]	He Z J, Liu Q, Lin S X. Improving statistical machine translation using lexicalized rule selection[C]. Manchester:International Conference on Coling, 2008.
[10]	段雨佳, 鞠婷. 基于深度学习的代码审查意见有效性评估[J]. 电子科技, 2020,33(1):39-45.
	Duan Yujia, Ju Ting. Evaluation of code review comments based on deep learning[J]. Electronic Science and Technology, 2020,33(1):39-45.
[11]	Yoshua B, Réjean D, Pascal V, et al. A neural probabilistic language model[J]. Journal of Machine Learning Research, 2003,(3):1137-1155.
[12]	缪广寒. 基于Word2vec和SVM的微博情感挖掘与仿真分析[J]. 电子科技, 2018,31(5):81-83.
	Miu Guanghan. Emotion mining and simulation analysis of microblogging based on Word2vec and SVM[J]. Electronic Science and Technology, 2018,31(5):81-83.
[13]	Sutskever I, Vinyals O, Le Q V V. Sequence to sequence learning with neural networks[C]. Montreal:Advances in Neural Information Processing Systems, 2014.
[14]	Hsieh Y L, Liu S H, Chen K Y, et al. Exploiting sequence-to-sequence generation framework for automatic abstractive summarization[C]. Tainan: Proceedings of the Twenty-eighth Conference on Computational Linguistics and Speech Processing, 2016.
[15]	尹光花, 刘小明, 张露, 等. 基于LSTM特征模板的短文本情感要素分析与研究[J]. 电子科技, 2018,31(11):38-41,46.
	Yin Guanghua, Liu Xiaoming, Zhang Lu, et al. Sentiment elements of internet short texts for analysis and research based on LSTM network mode[J]. Electronic Science and Technology, 2018,31(11):38-41,46.
[16]	曾浩, 尚维来. Python界面程序开发应用技术[J]. 科教文汇, 2010,(10):95-97.
	Zeng Hao, Shang Weilai. Development and application technology of Python interface program[J]. The Science Education Article Collects, 2010,(10):95-97.
[17]	Chen Q C, Hu B T, Zhu F Z. LCSTS:A large scale Chinese short text summarization dataset[C]. Lisbon:Conference on Empirical Methods in Natural Language Processing, 2015.
[18]	Lin C Y. ROUGE:A package for automatic evaluation of summaries[C]. Washington D.C.:Proceedings of the Workshop on Text Summarization Branches Out, 2004.
[19]	Erkan G, Radev D R. LexPageRank:prestige in multidocument text summarization[C]. Barcelona: Proceedings of the Conference on Empirical Methods on Natural Language Processing, 2004.
[20]	Radev D R, Blair G S, Zhang Z. Experiments in single and multi-document summarization using MEAD[J]. Ann Arbor, 2004,10(1):48109-48117.

参数名	数值
批训练尺寸Batch_size	64
神经网络每层神经元个数LSTM_units	512
Dropout策略失活比例Dropout_value	0.3
训练迭代次数Epochs	200
损失函数Loss	Cross_entropy
学习率Learning_rate	0.001
学习率衰减比例Learning_rate_decay	0.96/10 epochs
激活函数Activation_func	Softmax
优化器Optimizer	Adam(1×10^-3)

模型名	ROUGE-1指标	ROUGE-L指标
LexPageRank	23.63	18.92
MEAD	27.33	21.59
本文系统	29.91	24.68

Design of Text Title Generation Prototype System Based on Neural Network

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 20

Related Articles 15

Metrics

Comments

Recommended 0

[1]	LI Hui,WANG Yicheng. CNNCIFG-Attention Model for Text Sentiment Classifcation [J]. Electronic Science and Technology, 2022, 35(2): 46-51.
[2]	CHU Ping,NI Wei. Design of FPGA-Based SqueezeNet Inference Accelerator [J]. Electronic Science and Technology, 2022, 35(2): 20-26.
[3]	LU Wei,LIU Xiang,XUE Mian. Siamese Network Binocular Stereo Matching Based on Super-Pixel Segmentation [J]. Electronic Science and Technology, 2021, 34(9): 73-78.
[4]	ZHAN Yinwei,ZHU Baiwan,YANG Zhuo. Research and Application of Vehicle Face Recognition Algorithm [J]. Electronic Science and Technology, 2021, 34(8): 1-7.
[5]	LIAO Luming,ZHANG Wei. Batch Mixed Training Traffic Sign Recognition Based on Improved VGG16 Network [J]. Electronic Science and Technology, 2021, 34(8): 8-13.
[6]	SI Mingming,CHEN Wei,HU Chunyan,YIN Zhong. Fundus Blood Vessel Image Segmentation Combining Resnet50 and U-Net [J]. Electronic Science and Technology, 2021, 34(8): 19-24.
[7]	GU Wei,LI Feifei,CHEN Qiu. Pedestrian Detection Algorithm Based on Multiple Feature Fusion [J]. Electronic Science and Technology, 2021, 34(5): 29-34.
[8]	MENG Xiaojiao,ZHANG Shiwei,LI Xiaojian,LI Minyue,SONG Bingxin,LU Hongmin. An Evaluation Method of Vehicle-Mounted Communication Equipment Performance Based on BP Neural Network [J]. Electronic Science and Technology, 2021, 34(5): 24-28.
[9]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.
[10]	DOU Zhengli,WANG Yagang. Modeling of Combustion Chamber Temperature Model of Combined Cycle Unit Based on Elman Neural Network [J]. Electronic Science and Technology, 2021, 34(3): 60-64.
[11]	ZHU Bin,LIU Zilong. Image Classification Method Using Convolutional Neural Network Based on New Initial Module [J]. Electronic Science and Technology, 2021, 34(2): 52-56.
[12]	ZHOU Xiaobo,ZOU Renling,LU Xuhua,WANG Haibin,ZHANG Junxiang. Research Progress of Surface Electromyography Signal Classifier Based on Artificial Neural Network [J]. Electronic Science and Technology, 2021, 34(2): 62-67.
[13]	YAN Chao,SUN Zhanquan,TIAN Engang,ZHAO Yangyang,FAN Xiaoyan. Research Progress of Medical Image Segmentation Based on Deep Learning [J]. Electronic Science and Technology, 2021, 34(2): 7-11.
[14]	HE Zhonglin,ZHOU Ping. Research on Sorting of Retired Lithium Battery Based on Radial Basis Function Neural Network [J]. Electronic Science and Technology, 2021, 34(2): 38-44.
[15]	ZHU Chengming,WEI Yunbing,JIANG Chengcheng,ZHU Jian'an. Estimation Model of Wind Power Reserve Capacity Based on PSO-BP Neural Network [J]. Electronic Science and Technology, 2021, 34(12): 36-41.