一种新型高效的文库知识图谱实体关系抽取算法

doi:10.19665/j.issn1001-2400.2021.06.010

摘要/Abstract

摘要：

实体关系抽取旨在给定的一个句子中抽取两个实体之间的语义关系。实体关系抽取是信息抽取和自然语言处理中的一项基本而重要的任务。尽管近年来已出现了一些较好的深度实体关系抽取算法,但如何充分利用语料库信息并有效地抽取语句中实体间的语义关系,以进一步提高深度模型的精度仍面临着严峻的挑战。本文首先基于训练文库构建了一种新的实体语义关系图,随着测试的进行它也可以不断扩展。实体语义关系图用于从语料库的所有句子中全局获取实体之间的语义关系。然后,选取语料库中存在的大量“other”关系作为负样本进行训练,以提高分类性能。最后,利用轻量级预训框架ALBERT、图卷积网络和负样本学习三元组损失,提出了一种新的实体关系抽取算法。该算法能不断地汇总和完善与待抽取实体对间关系的相关知识,因此可以有效地提高实体关系抽取的精度。在SemEval-2010 Task8和TACRED的基准测试中对这种算法进行的广泛性能对比实验,表明该算法的性能均优于目前最具代表性的深度实体关系抽取算法。

关键词: 实体关系抽取, 自然语言处理, 图神经网络

Abstract:

Entity relation extraction aims to extract the semantic relation between two entities in a given sentence.Entity relation extraction is a basic and important task in information extraction and natural language processing.Although some good entity relation extraction deep learning algorithms have been presented,how to make full use of corpus information and extract the relationship between entities in a sentence effectively to further improve the accuracy of the model still faces challenges.In this paper,a new entity semantic relation graph is constructed based on the training corpus,which can be extended as the testing goes on.The entity semantic relation graph is used to globally capture the semantic relation correlations between entities from all the sentences in the corpuses.And then,a large number of “other” relations existing in the corpus are selected as negative samples to be trained to improve the classification performance.Finally,equipped with the light pre-trained ALBERT,a graph convolutional network,and the negative sample learning triplet loss,we present a new RE method,which can continuously summarize and perfect the knowledge related to the entity pairs to be extracted,and effectively improve the accuracy of entity relation extraction.Extensive experiments on the SemEval-2010 Task 8 and TACRED benchmark show that our proposed algorithm achieves a better performance than the competitive baselines.

Key words: entity relation extraction, natural language processing, graph neural network

中图分类号:

TP183

胡代旺,焦一源,李雁妮. 一种新型高效的文库知识图谱实体关系抽取算法[J]. 西安电子科技大学学报, 2021, 48(6): 75-83.

HU Daiwang,JIAO Yiyuan,LI Yanni. Novel and efficient algorithm for entity relation extraction with the corpus knowledge graph[J]. Journal of Xidian University, 2021, 48(6): 75-83.

图/表 5

图1

图2

表1

表2

表3

参考文献 28

[1]	ZENG D, LIU K, LAI S, et al. Relation Classification Via Convolutional Deep Neural Network[C]// Proceedings of COLING 2014,the 25th International Conference on Computational Linguistics:Technical Papers.Dublin: 2014:2335-2344.
[2]	XU Y, MOU L, LI G, et al. Classifying Relations Via Long Short Term Memory Networks Along Shortest Dependency Paths[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon:The Association for Computational Linguistics, 2015:1785-1794.
[3]	ZHOU P, SHI W, TIAN J, et al. Attention-Based Bidirectional Long Short-Term Memory Networks For Relation Classification[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin:The Association for Computer Linguistics, 2016:207-212.
[4]	DEVLIN J, CHANG M W, LEE K, et al. Bert:Pre-Training Of Deep Bidirectional Transformers For Language Understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies (NAACL).Minneapolis:Association for Computational Linguistics, 2019:4171-4186.
[5]	WU S, HE Y. Enriching Pre-Trained Language Model With Entity Information For Relation Classification[C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management.Beijing:ACM, 2019:2361-2364.
[6]	SOARES L B, FITZGERALD N, LING J, et al. Matching The Blanks:Distributional Similarity For Relation Learning[C]// Proceedings of the 57th Conference of the Association for Computational Linguistics.Florence:Association for Computational Linguistics, 2019:2895-2905.
[7]	MIWA M, BANSAL M. End-To-End Relation Extraction Using Lstms On Sequences And Tree Structures[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin:The Association for Computer Linguistics, 2016:1105-1116.
[8]	ZHANG Y, QI P, Manning C D. Graph Convolution Over Pruned Dependency Trees Improves Relation Extraction[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels:Association for Computational Linguistics, 2018:2205-2215.
[9]	GUO Z, ZHANG Y, LU W. Attention Guided Graph Convolutional Networks For Relation Extraction[C]// Proceedings of the 57th Conference of the Association for Computational Linguistics.Florence:Association for Computational Linguistics, 2019:241-251.
[10]	SUN K, ZHANG R, MAO Y, et al. Relation Extraction with Convolutional Network over Learnable Syntax-Transport Graph[C]// Proceedings of the 34th Association for the Advance of Artificial Intelligence.New York:AAAI, 2020:8928-8935.
[11]	ZHANG Z, SHU X, YU B, et al. Distilling Knowledge from Well-Informed Soft Labels for Neural Relation Extraction[C]// Proceedings of the 34th Association for the Advance of Artificial Intelligence.New York:AAAI, 2020:9620-9627.
[12]	JIN L, SONG L, ZHANG Y, et al. Relation Extraction Exploiting Full Dependency Forests[C]// Proceedings of the 34th Association for the Advance of Artificial Intelligence.New York:AAAI, 2020:8034-8041.
[13]	VEYSEH A P B, DERNONCOURT F, DOU D, et al. Exploiting the Syntax-Model Consistency for Neural Relation Extraction[EB/OL]. [2020-7-5]https://aclanthology.org/2020.acl-main.715.pdf .
[14]	GUO Z, NAN G, LU W, et al. Learning Latent Forests For Medical Relation Extraction[EB/OL]. [2020-1-7]https://www.ijcai.org/Proceedings/2020/0505.pdf .
[15]	VEYSEH A P B, DERNONCOURT F, THAI M T, et al. Multi-View Consistency for Relation Extraction via Mutual Information and Structure Prediction[C]// Proceedings of the 34th Association for the Advance of Artificial Intelligence.New York:AAAI, 2020:9106-9113.
[16]	ZHU H, LIN Y, LIU Z, et al. Graph Neural Networks With Generated Parameters For Relation Extraction[C]// Proceedings of the 57th Conference of the Association for Computational Linguistics.Florence:Association for Computational Linguistics, 2019:1331-1339.
[17]	CHRISTOPOULOU F, MIWA M, ANANIADOU S. A Walk-Based Model On Entity Graphs For Relation Extraction[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Melbourne:Association for Computational Linguistics, 2018:81-88.
[18]	NAN G, GUO Z, SEKULIĆ I, et al. Reasoning With Latent Structure Refinement For Document-Level Relation Extraction[EB/OL]. [2020-6-28]https://arxiv.org/pdf/2005.06312.pdf .
[19]	ZHAO Y, WAN H, GAO J, et al. Improving Relation Classification by Entity Pair Graph[C]// Proceedings of the 11th Asian Conference on Machine Learning.Nagoya:PMLR, 2019:1156-1171.
[20]	CHI P H, CHUNG P H, WU T H, et al. Audio albert:A Lite Bert For Self-Supervised Learning Of Audio Representation[C]// 2021 IEEE Spoken Language Technology Workshop.Piscataway:IEEE, 2021:344-350.
[21]	HENDRICKX I, KIM S N, KOZAREVA Z, et al. Semeval-2010 Task 8:Multi-Way Classification Of Semantic Relations Between Pairs Of Nominals[C]// Proceedings of the 5th International Workshop on Semantic Evaluation.Uppsala:The Association for Computer Linguistics, 2010:33-38.
[22]	ZHANG Y, ZHONG V, CHEN D, et al. Position-Aware Attention And Supervised Data Improve Slot Filling[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP).Copenhagen:Association for Computational Linguistics, 2017:35-45.
[23]	KIPF T N, WELLING M. Semi-Supervised Classification With Graph Convolutional Networks[C]// Proceedings of the 5th International Conference on Learning Representations.Toulon:OpenReview.net, 2017.
[24]	ZHANG Z, HAN X, LIU Z, et al. ERNIE:Enhanced Language Representation with Informative Entities[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Florence:Association for Computational Linguistics, 2019:1441-1451.
[25]	PETERS M E, NEUMANN M, LOGAN IV R L, et al. Knowledge Enhanced Contextual Word Representations[C]// Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing.Hong Kong:Association for Computational Linguistics, 2019:43-54.
[26]	MISHCHUK A, MISHKIN D, RADENOVIC F, et al. Working hard to know your neighbor's margins:Local descriptor learning loss[EB/OL]. [2018-1-12]https://arxiv.org/pdf/1705.10872.pdf .
[27]	LI X, JIA X, JING X Y. Negative-Aware Training:Be Aware of Negative Samples[EB/OL]. [2019-06-08]https://ebooks.iospress.nl/pdf/doi/10.3233/FAIA200228 .
[28]	SCHROFF F, KALENICHENKO D, PHILBIN J. Facenet:A Unified Embedding For Face Recognition And Clustering[C]// Proceedings of the IEEE conference on computer vision and pattern recognition.Piscataway:IEEE, 2015:815-823.

算法名称	F₁/(%)
C-AGGCN^[9](2019)	85.7
MTB+ BERTEM^[6](2019)	89.5
LST-AGCN^[10](2020)	86.0
Student-R^[11](2020)	86.8
MVC^[15](2020)	86.1
ERECKG (ours)	91.4

算法名称	F₁/(%)
PA-LSTM^[22](2017)	65.1
C-GCN^[8](2018)	66.4
C-AGGCN^[9](2019)	69.0
MTB+ BERTEM^[6](2019)	71.5
LST-AGCN^[10](2020)	68.8
ERECKG (ours)	74.7

方案	SemEval-2010Task 8	TACRED/(%)
方案1	85.5	71.8
方案2	86.8	73.9
方案3	86.2	73.4
方案4	87.4	74.7