短文本新闻标题生成方法

doi:10.16180/j.cnki.issn1007-7820.2024.09.013

Abstract

Abstract:

Today's news has the characteristics of short text, frequent release, timeliness, etc. A media account releases dozens of news in a day. Developing suitable and attractive headlines for large volumes of news has become a major part of the work of media workers. Media workers need a system that automatically generates short text headlines to relieve their stress. To solve this problem, this study proposes a short text news title generation model. The model adopts sequence-to-sequence structure, using pre-trained language model and layered self-attention decoder in encoder and decoder respectively. In order to make the generated headlines contain the key information of the original news, a staged training method based on LCSTS data set and Weibo4 data set is proposed, and the model learns to extract the key news information and construct a stylized expression from the two data sets respectively, so that the generated headlines can accurately express the core content of the news and attract readers.

Key words: news headline generation, pre-training language model,, layered self-attention decoder, encoder, text extraction, text generation

CLC Number:

TP391

ZHAO Ming. Researchon Short Text News Title Generation Method[J].Electronic Science and Technology, 2024, 37(9): 87-94.

Figures/Tables 8

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Table 1.

Table 2.

Table 3.

Figure 5.

References 20

[1]	汪家伟, 余晓. 基于深度学习的文本分类研究综述[J]. 电子科技, 2024, 37(1):81-86.
	Wang Jiawei, Yu Xiao. Review of text classification research based on deep learning[J]. Electronic Science and Technology, 2024, 37(1):81-86.
[2]	Devlin J, Chang MW, Lee K. BERT:Pre-training of deep bidirectional transformers for language understanding[C]. Minneapolis:Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[3]	Vaswani A, Shazeer N, Parmar N. Attention is all you need[C]. Long Beach: Proceedings of the Thirty-first International Conference on Neural Information Processing Systems, 2017:6000-6010.
[4]	张琪, 范永胜. 基于改进T5 PEGASUS模型的新闻文本摘要生成[J]. 电子科技, 2023, 36(12):72-78.
	Zhang Qi, Fan Yongsheng. Research on generating news text summarrization based on improved T5 PEGASUS model[J]. Electronic Science and Technology, 2023, 36(12):72-78.
[5]	Mihalcea R, Tarau P. Textrank:Bringing order into text[C]. Doha: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2004:404-411.
[6]	Page L, Brin S, Motwani R. The PageRank citation ranking:Bringing order to the web[M]. Palo Alto: Stanford InfoLab, 1999:61-89.
[7]	何传鹏, 尹玲, 黄勃, 等. 基于BERT和LightGBM的文本关键词提取方法[J]. 电子科技, 2023, 36(3):7-13.
	He Chuanpeng, Yin Ling, Huang Bo, et al. Text keyword extraction method based on BERT and LightGBM[J]. Electronic Science and Technology, 2023, 36(3):7-13.
[8]	Cheng J, Lapata M. Neural summarization by extracting sentences and words[C]. Berlin:Proceedings of the Fifty-fourth Annual Meeting of the Association for Computational Linguistics, 2016:484-494.
[9]	Nallapati R, Zhai F, Zhou B. Summarunner:A recurrent neural network based sequence model for extractive summarization of documents[C]. San Francisco: Proceedings of the AAAI Conference on Artificial Intelligence, 2017:311-318.
[10]	Narayan S, Cohen S B, Lapata M. Ranking sentences for extractive summarization with reinforcement learning[C]. New Orleans:Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018:1747-1759.
[11]	Zhou L, Hovy E. Template-filtered headline summarization[C]. Barcelona: Proceedings of the Workshop on Text Summarization Branches Out, 2004:56-60.
[12]	Rush A M, Chopra S, Weston J. A neural attention model for abstractive sentence summarization[C]. Lisbon: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015:379-389.
[13]	李志欣, 彭智, 唐素勤, 等. 融合上下文信息和关键信息的文本摘要[J]. 中文信息学报, 2022, 36(1): 83-91.
	Li Zhixin, Peng Zhi, Tang Suqin, et al. Fusing context information and key information for text summarization[J]. Journal of Chinese Information Processing, 2022, 36(1):83-91.
[14]	See A, Liu P J, Manning C D. Get to the point:Summarization with pointer-generator networks[C]. Vancouver: Proceedings of the Fifty-fifth Annual Meeting of the Association for Computational Linguistics, 2017:1073-1083.
[15]	Lin J, Sun X, Ma S. Global encoding for abstractive summarization[C]. Melbourne: Proceedings of the Fifty-sixth Annual Meeting of the Association for Computational Linguistics, 2018:163-169.
[16]	Mikolov T, Sutskever I, Chen K. Distributed representations of words and phrases and their compositionality[C]. Nevada: Proceedings of the Twenty-sixth International Conference on Neural Information Processing Systems, 2013:3111-3119.
[17]	Pennington J, Socher R, Manning C D. Glove:Global vectors for word representation[C]. Doha: Proceedings of the Conference on Empirical Mmethods in Natural Language Processing, 2014:1532-1543.
[18]	Peters M, Neumann M, Iyyer M. Deep contextualized word representations[C]. New Orleans:Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018:2227-2237.
[19]	Xu Yueshen. Collaborative APIs recommendation for artificial intelligence of things with information fusion[J]. Future Generation Computer Systems, 2021, 12(5):471-479.
[20]	Loshchilov I, Hutter F. Decoupled weight decay regularization[C]. Vancouver: International Conference on Learning Representations, 2018:452-459.

模型	R-1	R-2	R-L
RNN	21.50	8.90	18.60
RNN-Context	29.90	17.40	27.20
CopyNet	34.40	21.60	31.30
Seq2Seq+CGU	39.40	26.90	36.50
BERT+TransformerDecoder	40.48	25.37	38.08

模型	R-1	R-2	R-L
①BERT(all-freezed)+ TransformerDecoder	34.35	19.93	31.32
②BERT(two-layer-unfreezed)+ TransformerDecoder	35.18	20.48	32.90
③BERT(all-unfreezed)+ TransformerDecoder	40.48	25.37	38.08

	Rouge_LCSTS			Rouge_Weibo4			Rouge_AVG
	R-1	R-2	R-L	R-1	R-2	R-L	R-1	R-2	R-L
①Weibo4_only	28.68	13.47	25.91	44.92	31.17	42.77	36.80	22.32	34.34
②LCSTS_only	40.48	25.37	38.08	39.25	25.61	37.61	39.87	25.49	37.85
③Stagewise	40.02	25.24	37.93	41.84	27.64	39.99	40.93	26.44	38.96

Researchon Short Text News Title Generation Method

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 20

Related Articles 15

Metrics

Comments

Recommended 0

[1]	LI Qing, WAN Weibing. Automatic Summarization of Small Samples Based on Enhanced Regularization [J]. Electronic Science and Technology, 2024, 37(7): 16-24.
[2]	PANG Jiangfei, SUN Zhanquan. Multi-Encoder Transformer for End-to-End Speech Recognition [J]. Electronic Science and Technology, 2024, 37(4): 1-7.
[3]	BAI Yingqi,PALIDAN·Tuerxun . A Scientific Literature Recommendation Method Based on Multi-Task Learning [J]. Electronic Science and Technology, 2023, 36(4): 59-64.
[4]	WU Xu,LIU Xiang. Copy-Move Forgery Detection Algorithm Based on Non-Local Self-Correlation [J]. Electronic Science and Technology, 2022, 35(10): 59-64.
[5]	GONG Jialiang,TANG Qingshan,BAI Chuang. Design of Efficiency Parallel HDMI 2.0 Encoder Based on FPGA [J]. Electronic Science and Technology, 2021, 34(4): 34-40.
[6]	ZHENG Zhanpeng,YIN Zhong. Inter-Subject Recognition of Cognitive Workload Based on Ensemble SDAE and EEG [J]. Electronic Science and Technology, 2021, 34(3): 48-53.
[7]	ZHANG Can,CHEN Wei,YIN Zhong. Semantic Segmentation of Cervical Cell Image Based on Weak Supervision [J]. Electronic Science and Technology, 2021, 34(12): 68-74.
[8]	LI Fupeng,FU Dongxiang. Sentiment Analysis Method of Financial Text Based on Transformer Encoder [J]. Electronic Science and Technology, 2020, 33(9): 10-15.
[9]	ZHAO Yihe,SHAO Jie,CHENG Yongliang. Behavior Modeling of Class-D Power Amplifier Based on Encoder-Decoder Model [J]. Electronic Science and Technology, 2020, 33(2): 20-24.
[10]	ZHENG Meng. Research on Intelligent English Translation Method Based on Improved Attention Mechanism Model [J]. Electronic Science and Technology, 2020, 33(11): 84-87.
[11]	BEI Chenyuan,YU Haibin,PAN Mian,JIANG Jie,LÜ Bingyun. Gland Cell Image Segmentation Algorithm Based on Improved U-Net Network [J]. Electronic Science and Technology, 2019, 32(11): 18-22.
[12]	SHI Ce,QIU Jiong. The Fire Protocol Intelligent Analysis Model Based on Deep Learning [J]. Electronic Science and Technology, 2019, 32(1): 80-85.
[13]	XIE Lin,LI Feifei,CHEN Qiu. Scene Recognition Algorithm Based on Sparse Autoencoder [J]. Electronic Science and Technology, 2019, 32(1): 38-41.
[14]	YIN Zheng,TANG Chunhui,ZHANG Xuanxiong. Image Recognition Based on Improved Sparse Auto-encoder [J]. , 2016, 29(1): 124-.
[15]	WEI Xiangyi,DING Hongchang,CAO Guohua. Current Status and Development of Detection of Photoelectric Encoders [J]. , 2015, 28(9): 184-.