基于神经网络的文本标题生成原型系统设计

doi:10.16180/j.cnki.issn1007-7820.2021.05.007

摘要/Abstract

摘要：

针对传统人工总结、编写标题的方法在耗费大量人力、时间成本的同时难以应对互联网中大量不规范的文本的问题,文中设计了一种基于神经网络的文本标题生成原型系统。在文本标题生成原型系统中通过基于神经网络编码器-解码器模型对文本进行建模计算,从而经济、高效地生成一条准确、简洁、切合原文的标题。在编码器部分采用双向长短期记忆神经网络,充分利用上下文之间的语义联系。解码器部分则采用单向神经网络进行解码操作,并引入注意力机制来缓解信息丢失,提高标题生成效果。在LCSTS数据集上进行实验得到ROUGE-1、ROUGE-L评价指标分别为29.91和24.68,证明了该标题生成原型系统的有效性。

关键词: 人工智能, 自然语言处理, 神经网络, 标题生成, 原型系统, 词向量, 注意力机制, 生成式技术

Abstract:

In view of the traditional manual methods cost a lot of manpower and time and can not deal with the problem of massive of non-standard texts, a prototype system of generating text titles is designed in the proposed study. In the prototype system, the non-standard text is calculated by the encoder-decoder model which is based on neural network to produce an accurate title. In the encoder part, the bidirectional long short-term memory neural network is adopted to make full use of the semantic connection between contexts. In the decoder part, one-way neural network is used for decoding operation, and attention mechanism is added to alleviate information loss and improve the effect of title generation. The evaluation indexes of ROUGE-1 and ROUGE-L obtained by experiments on LCSTS data set are 29.91 and 24.68, proving the effectiveness of the title generation prototype system.

Key words: artificial intelligence, natural language processing, neural network, title generation, prototype system, word vector, attention mechanism, generative technology

中图分类号:

TP391

张仕森,孙宪坤,尹玲,李世玺. 基于神经网络的文本标题生成原型系统设计[J]. 电子科技, 2021, 34(5): 35-41.

ZHANG Shisen,SUN Xiankun,YIN Ling,LI Shixi. Design of Text Title Generation Prototype System Based on Neural Network[J]. Electronic Science and Technology, 2021, 34(5): 35-41.

图/表 12

图1

图2

图3

图4

表1

图5

图6

表2

图7

图8

图9

表3

参考文献 20

[1]	Aone C, Okurowski M E, Gorlinsky J, et al. A trainable summarizer with knowledge acquired from robust NLP techniques[C]. Montreal:The Thirty-sixth Annual Meeting of the Association for Computational Linguistics, 1999.
[2]	Lin C Y. Training a selection function for extraction[C]. New York:The Eighth International Conference on Information and Knowledge Management, 1999.
[3]	Mihalcea R, Tarau P. TextRank:bringing order into texts[C]. Barcelona:Conference on Empirical Methods in Natural Language Processing, 2004.
[4]	Erkan, Radev. Dragomir R.LexRank: graph-based lexical centrality as salience in text summarization[J]. Journal of Artificial Intelligence Research, 2004,22(1):457-479. doi: 10.1613/jair.1523
[5]	Lopyrev K. Generating news headlines with recurrent neural networks[J]. Computer Science, 2015,2(2):159-165.
[6]	黄鑫池. 神经网络自动生成汉语新闻标题的应用[J]. 电子技术与软件工程, 2018(22):134-135.
	Huang Xinchi. Application of neural network to automatically generate Chinese news headlines[J]. Electronic Technology & Software Engineering, 2018(22):134-135.
[7]	Hou L W, Hu P, Chao B. Abstractive document summarization via neural model with joint attention[C]. Dalian:Proceedings of the Sixth Conference on Natural Language Processing and Chinese Computing, 2017.
[8]	Ranzato M A, Chopra S, Auli M, et al. Sequence level training with recurrent neural networks[J/OL]. (2016-05-06)[2019-11-28] .
[9]	He Z J, Liu Q, Lin S X. Improving statistical machine translation using lexicalized rule selection[C]. Manchester:International Conference on Coling, 2008.
[10]	段雨佳, 鞠婷. 基于深度学习的代码审查意见有效性评估[J]. 电子科技, 2020,33(1):39-45.
	Duan Yujia, Ju Ting. Evaluation of code review comments based on deep learning[J]. Electronic Science and Technology, 2020,33(1):39-45.
[11]	Yoshua B, Réjean D, Pascal V, et al. A neural probabilistic language model[J]. Journal of Machine Learning Research, 2003,(3):1137-1155.
[12]	缪广寒. 基于Word2vec和SVM的微博情感挖掘与仿真分析[J]. 电子科技, 2018,31(5):81-83.
	Miu Guanghan. Emotion mining and simulation analysis of microblogging based on Word2vec and SVM[J]. Electronic Science and Technology, 2018,31(5):81-83.
[13]	Sutskever I, Vinyals O, Le Q V V. Sequence to sequence learning with neural networks[C]. Montreal:Advances in Neural Information Processing Systems, 2014.
[14]	Hsieh Y L, Liu S H, Chen K Y, et al. Exploiting sequence-to-sequence generation framework for automatic abstractive summarization[C]. Tainan: Proceedings of the Twenty-eighth Conference on Computational Linguistics and Speech Processing, 2016.
[15]	尹光花, 刘小明, 张露, 等. 基于LSTM特征模板的短文本情感要素分析与研究[J]. 电子科技, 2018,31(11):38-41,46.
	Yin Guanghua, Liu Xiaoming, Zhang Lu, et al. Sentiment elements of internet short texts for analysis and research based on LSTM network mode[J]. Electronic Science and Technology, 2018,31(11):38-41,46.
[16]	曾浩, 尚维来. Python界面程序开发应用技术[J]. 科教文汇, 2010,(10):95-97.
	Zeng Hao, Shang Weilai. Development and application technology of Python interface program[J]. The Science Education Article Collects, 2010,(10):95-97.
[17]	Chen Q C, Hu B T, Zhu F Z. LCSTS:A large scale Chinese short text summarization dataset[C]. Lisbon:Conference on Empirical Methods in Natural Language Processing, 2015.
[18]	Lin C Y. ROUGE:A package for automatic evaluation of summaries[C]. Washington D.C.:Proceedings of the Workshop on Text Summarization Branches Out, 2004.
[19]	Erkan G, Radev D R. LexPageRank:prestige in multidocument text summarization[C]. Barcelona: Proceedings of the Conference on Empirical Methods on Natural Language Processing, 2004.
[20]	Radev D R, Blair G S, Zhang Z. Experiments in single and multi-document summarization using MEAD[J]. Ann Arbor, 2004,10(1):48109-48117.

参数名	数值
批训练尺寸Batch_size	64
神经网络每层神经元个数LSTM_units	512
Dropout策略失活比例Dropout_value	0.3
训练迭代次数Epochs	200
损失函数Loss	Cross_entropy
学习率Learning_rate	0.001
学习率衰减比例Learning_rate_decay	0.96/10 epochs
激活函数Activation_func	Softmax
优化器Optimizer	Adam(1×10^-3)

模型名	ROUGE-1指标	ROUGE-L指标
LexPageRank	23.63	18.92
MEAD	27.33	21.59
本文系统	29.91	24.68