融合自注意力机制与CNN-BiGRU的事件检测

doi:10.19665/j.issn1001-2400.2022.05.021

摘要/Abstract

摘要：

基于卷积神经网络和循环神经网络的事件检测已得到广泛研究。然而卷积神经网络只能考虑卷积窗口内的局部信息,忽略了词语的上下文联系。循环神经网络存在梯度消失和短期记忆的问题,且其变体门控循环单元无法得到每个词语的特征。为此,提出一种基于自注意力机制与卷积双向门控循环单元模型的事件检测方法。该模型将词向量和位置向量作为输入,不仅能够通过卷积操作提取不同粒度的词汇级特征,通过双向门控循环单元提取句子级特征,而且通过自注意力机制考虑全局信息,关注对事件检测更重要的特征。将提取的词汇级特征和句子级特征拼接作为联合特征,通过softmax分类器进行候选词分类,从而完成事件检测任务。实验结果显示,在ACE2005英文语料上,事件检测中触发词识别和分类的F值分别达到78.9%和76.0%,优于基线事件检测方法的结果,且模型表现出更好的收敛性。实验结果表明,所提出的基于自注意力机制与卷积双向门控循环单元模型有良好文本特征提取能力,提升了事件检测的性能。

关键词: 事件检测, 信息抽取, 卷积神经网络, 双向门控循环单元, 自注意力机制

Abstract:

Event detection methods based on convolutional neural networks and recurrent neural networks have been widely investigated.However,convolutional neural networks only consider local information within the convolution window,ignoring the context of words.Recurrent neural networks have the problem of vanishing gradient and short-term memory,and their variant gated recurrent units cannot get the features of each word.Therefore,in this paper,an event detection method based on self-attention and convolutional bidirectional gated recurrent units model is proposed,which takes both word vectors and position vectors as inputs.It can not only extract vocabulary level features with different granularities by convolutional neural network and sentence level features by bidirectional gated recurrent units,but also consider global information and pay attention to more important features for event detection by self-attention.The extracted lexical-level features and sentence-level features are combined as the joint features,and the candidate words are classified by the softmax classifier to complete the event detection task.Experimental results show that the F scores of trigger words recognition and classification reach 78.9% and 76.0% respectively on the ACE2005 English corpus,which are better than the results of benchmark methods.Furthermore,the model shows great convergence.It is shown that the proposed model based on self-attention and convolutional bidirectional gated recurrent units possesses good ability of text feature extraction and improves the performance of event detection.

Key words: event detection, information extraction, convolutional neural networks, bidirectional gated recurrent unit, self-attention mechanism

中图分类号:

TP18

王侃, 王孟洋, 刘鑫, 田国强, 李川, 刘伟. 融合自注意力机制与CNN-BiGRU的事件检测[J]. 西安电子科技大学学报, 2022, 49(5): 181-188.

WANG Kan, WANG Mengyang, LIU Xin, TIAN Guoqiang, LI Chuan, LIU Wei. Event detection by combining self-attention and CNN-BiGRU[J]. Journal of Xidian University, 2022, 49(5): 181-188.

图/表 10

图1

图2

图3

图4

图5

表1

图6

图7

表2

图8

参考文献 19

[1]	WEI H, ZHOU A, ZHANG Y J. Biomedical Event Trigger Extraction Based on Multi-Layer Residual BiLSTM and Contextualized Word Representations[J]. International Journal of Machine Learning and Cybernetics, 2021, 12(18):1-13. doi: 10.1007/s13042-020-01103-9
[2]	XIANG W, WANG B. A Survey of Event Extraction From Text[J]. IEEE Access, 2019, 7:173111-173137. doi: 10.1109/ACCESS.2019.2956831
[3]	万齐智, 万常选, 胡蓉, 等. 基于句法语义依存分析的中文金融事件抽取[J]. 计算机学报, 2021, 44(3):508-530.
	WAN Qizhi, WAN Changxuan, HU Rong, et al. Chinese Financial Event Extraction Based on Syntactic and Semantic Dependency Parsing[J]. Chinese Journal of Computers, 2021, 44(3):508-530.
[4]	YU X Y, RONG W G, ZHOU D.Y, et al. LSTM-Based End-to-End Framework for Biomedical Event Extraction[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2020, 17(6):2029-2039. doi: 10.1109/TCBB.2019.2916346
[5]	贺瑞芳, 段绍杨. 基于多任务学习的中文事件抽取联合模型[J]. 软件学报, 2019, 30(4):1015-1030.
	HE Ruifang, DUAN Shaoyang. Joint Chinese Event Extraction Based Multi-task Learning[J]. Journal of Software, 2019, 30(4):1015-1030.
[6]	沈兰奔, 武志昊, 纪宇泽, 等. 结合注意力机制与双向LSTM的中文事件检测方法[J]. 中文信息学报, 2019, 33(9):79-87.
	SHEN Lanben, WU Zhihao, JI Yuze, et al. Chinese Event Detection Method Combining Attention Mechanism and BiLSTM[J]. Journal of Chinese Information Processing, 2019, 33(9):79-87.
[7]	ZHAN L Y, JIANG X P, LIU Q. Research on Chinese Event Extraction Method Based on HMM and Multi-stage Method[J]. Journal of Physics:Conference Series, 2021, 1732(1):1-4.
[8]	AHN D. The Stage of Event Extraction[C]// Proceedings of the Workshop on Annotating and Reasoning about Time and Events. Sydney: Association for Computational Linguistics, 2006:1-8.
[9]	LI Q, JI H, HUANG L. Joint Event Extraction via Structured Prediction with Global Features[C]// Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Sofia: Association for Computational Linguistics, 2013:73-82.
[10]	CHEN Y B, XU L H, LIU K, et al. Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Beijing: Association for Computational Linguistics, 2015:167-176.
[11]	NGUYEN T H, CHO K, GRISHMAN R. Joint Event Extraction via Recurrent Neural Networks[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. San Diego: Association for Computational Linguistics, 2016:300-309.
[12]	FENG X C, QIN B, LIU T. A Language-Independent Neural Network for Event Detection[J]. Science China Information Sciences, 2018, 61(9):81-92.
[13]	苗佳, 段跃兴, 张月琴, 等. 基于CNN-BiGRU模型的事件触发词抽取方法[J]. 计算机工程, 2021, 47(9):69-74.
	MIAO Jia, DUAN Yuexing,ZHANG Yueqin.Event Trigger Word Extraction Method Based on CNN-BiGRU Model[J]. Computer Engineering, 2021, 47(9):69-74.
[14]	VASWANI A, SHAZEER N, PARMAR, et al. Attention is All You Need[C]// Proceedings of the 31st Conference on Neural Information Processing Systems. California: Curran Associates Inc, 2017:5998-6008.
[15]	WEI Y, ZHOU L F, QIAN Z J, et al. Sentiment Analysis of Student Texts Using the CNN-BiGRU-AT Model[J]. Scientific Programming, 2021, 2021(9):1-9.
[16]	CHEN N, LI J, MAN Y. Research on Aviation Unsafe Information Text Analysis Based on Improved CNN-BiGRU-att Model[C]// Proceedings of 2021 IEEE 3rd International Conference on Civil Aviation Safety and Information Technology. Piscataway: IEEE, 2021:322-327.
[17]	曹卫东, 李嘉琪, 王怀超. 采用注意力门控卷积网络模型的目标情感分析[J]. 西安电子科技大学学报, 2019, 46(6):30-36.
	CAO Weidong, LI Jiaqi, WANG Huaichao. Analysis of Targeted Sentiment by the Attention Gated Convolutional Network Model[J]. Journal of Xidian University, 2019, 46(6):30-36.
[18]	李源, 崔玉爽, 王伟. 一种基于字词双通道网络的文本情感分析方法[J]. 西安电子科技大学学报, 2021, 48(6):179-186.
	LI Yuan, CUI Yushuang, WANG Wei. Method for the Analysis of Text Sentiment Based on the Word Dual-Channel Network[J]. Journal of Xidian University, 2021, 48(6):179-186.
[19]	ZHANG Z K, XU W, CHEN Q. Joint Event Extraction Based on Skip-Window Convolutional Neural Networks[C]// Proceedings of the 5th CCF Conference on Natural Language Processing and Chinese Computing & the 24rth International Conference on Computer Processing of Oriental Languages. Kunming: Springer International Publishing, 2016:324-334.

方法	触发词识别			触发词分类
方法	准确率(P)	召回率(R)	F值	准确率(P)	召回率(R)	F值
MaxENT	76.2	60.5	67.4	74.5	59.1	65.9
Li’s structure	76.9	65.0	70.4	73.7	62.3	67.5
DMCNN	80.4	67.7	73.5	75.6	63.6	69.1
JRNN	68.5	75.7	71.9	66.0	73.0	69.3
SCNN	78.1	71.8	74.8	74.1	64.8	69.1
Att-CNN-BiGRU	82.6	76.0	78.9	76.5	75.8	76.0

方法	触发词识别			触发词分类
方法	准确率(P)	召回率(R)	F值	准确率(P)	召回率(R)	F值
CNN-BiGRU	80.7	73.8	76.9	72.5	69.5	70.9
Att-CNN-BiGRU	82.6	76.0	78.9	76.5	75.8	76.0