基于字符增强的工业设备故障命名实体识别

doi:10.16180/j.cnki.issn1007-7820.2024.10.007

摘要/Abstract

摘要：

针对工业设备故障领域训练数据少、实体结构复杂和实体分布不均匀等问题,文中构建了工业设备故障命名实体识别语料库。为解决字符级命名实体识别模型难以表示工业设备故障领域的专业词汇信息问题,文中提出一种基于字符增强的工业设备故障命名实体识别模型。在嵌入层,直接在RoBERTa-WWM(Robustly Optimized BERT Pretraining Approach with Whole Word Masking)的Transformer层之间融入专业词汇信息,将单词信息分配给其包含的每个字来达到增强语义的目的,通过BiLSTM(Bidirectional Long Short-Term Memory)获得全局语义信息,利用CRF(Conditional Random Field)学习相邻标签之间的依赖关系,以获得最佳句子级标签序列。实验结果证明,所提模型对工业设备故障命名实体识别任务具有良好的效果,平均F1值达到了92.403%。

关键词: 工业设备故障, 语料库, 命名实体识别, RoBERTa-WWM, 专业词向量, BiLSTM, CRF, 深度学习

Abstract:

To address the issues of sparse training data, complex entity structures, and uneven entity distribution in the industrial equipment failure domain, this study constructs an industrial equipment failure named entity recognition corpus. Due to the difficulty of character-level named entity recognition models in representing the professional vocabulary information in the field of industrial equipment failure, this study proposes a character-enhanced industrial equipment failure named entity recognition model to address this problem. In the embedding layer, professional vocabulary information is directly fused between the Transformer layers of RoBERTa-WWM (Robustly Optimized BERT Pretraining Approach with Whole Word Masking) to allocate word information to each of its constituent characters for enhanced semantics. The global semantic information is obtained through a BiLSTM(Bidirectional Long Short-Term Memory), and the CRF(Conditional Random Field) is used to learn the dependency relationship between adjacent labels to obtain the optimal sentence-level label sequence. Experimental results demonstrate that the proposed model has good performance on industrial equipment fault named entity recognition tasks, with an average F1 score of 92.403%.

Key words: industrial equipment failure, corpus, named entity recognition, RoBERTa-WWM, professional word embedding, BiLSTM, CRF, deep learning

中图分类号:

TP183

张阳, 刘瑾. 基于字符增强的工业设备故障命名实体识别[J]. 电子科技, 2024, 37(10): 48-54.

ZHANG Yang, LIU Jin. Character Enhancement Based on Named Entity Recognition for Industrial Equipment Faults[J]. Electronic Science and Technology, 2024, 37(10): 48-54.

图/表 10

图1

图2

图3

图4

图5

表1

表2

表3

图6

图7

参考文献 19

[1]	王骏东, 杨军, 裴洋舟, 等. 基于知识图谱的配电网故障辅助决策研究[J]. 电网技术, 2021, 45(6):2101-2112.
	Wang Jundong, Yang Jun, Pei Yangzhou, et al. Distribution network fault assistant decision-making based on knowledge graph[J]. Power System Technology, 2021, 45(6):2101-2112.
[2]	Martinez-Gil J, Buchgeher G, Gabauer D, et al. Root cause analysis in the industrial domain using knowledge graphs:A case study on power transformers[J]. Procedia Computer Science, 2022, 20(8):944-953.
[3]	Zhang Y, Kang J, Dai W. Non-functional requirements e-licitation based on domain knowledge graph for automatic code generation of industrial cyber-physical systems[C]. Toronto: IECON the Forty-seventh Annual Conference of the IEEE Industrial Electronics Society,2021:1-6.
[4]	Chen X, Ouyang C, Liu Y, et al. Improving the named e-ntity recognition of Chinese electronic medical records by combining domain dictionary and rules[J]. International Journal of Environmental Research and Public Health, 2020, 17(8):2687-2703.
[5]	Feng X, Li Y, Hang Z, et al. TBR-NER:Research on CO-VID-19 text information extraction based on joint learning of topic recognition and named entity recognition[J]. Journal of Sensors, 2022, 20(22):1-15.
[6]	杨鹤, 于红, 刘巨升, 等. 基于BERT+BiLSTM+CRF深度学习模型和多元组合数据增广的渔业标准命名实体识别[J]. 大连海洋大学学报, 2021, 36(4):661-669.
	Yang He, Yu Hong, Liu Jusheng, et al. Fishery standard named entity recognition based on BERT+BiLSTM+C-RF deep learning model and multivariate combined data augmentation[J]. Journal of Dalian Fisheries University, 2021, 36(4):661-669.
[7]	Ćeović H, Kurdija A S, Delać G, et al. Named entity recognition for addresses:An empirical study[J]. IEEE Access, 2022(10):42108-42120.
[8]	孟昕. 基于深度学习的法律文书识别方法研究[J]. 电子科技, 2019, 32(12):84-86.
	Meng Xin. Research on recognition method of legal documents based on deep learning[J]. Electronic Science and Technology, 2019, 32(12):84-86.
[9]	Zhang H, Wang X, Liu J, et al. Chinese named entity recognition method for the finance domain based on enhanced features and pretrained language models[J]. Information Sciences, 2023, 62(5):385-400.
[10]	Brandsen A, Verberne S, Lambers K, et al. Can BERT dig it?Named entity recognition for information retrieval in the archaeology domain[J]. Journal on Computing and Cultural Heritage, 2022, 15(3):1-18.
[11]	Zheng K, Sun L, Wang X, et al. Named entity recognition in electric power metering domain based on attention mechanism[J]. IEEE Access, 2021(9):152564-152573.
[12]	Ivanin V, Artemova E, Batura T, et al. RuREBus:A case study of joint named entity recognition and relation extraction from E-government domain[C]. Skolkovo:Analysis of Images, Social Networks and Texts: The Ninth International Conference,2020:19-27.
[13]	Li J, Sun A, Han J, et al. A survey on deep learning for named entity recognition[J]. IEEE Transactions on Knowledge and Data Engineering, 2020, 34(1):50-70.
[14]	Devlin J, Chang M, Lee K, et al. BERT:Pretraining of deep bidirectional transformers for language understanding[C]. Minneapolis:Proceedings of the Seventeenth Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2019:4171-4186.
[15]	Hochreiter S, Schmidhuber J. Long shortterm memory[J]. Neural Computation, 1997, 9(8):1735-1780. doi: 10.1162/neco.1997.9.8.1735 pmid: 9377276
[16]	Sa'a D A A, Tawalbeh S K, Al-Smadi M, et al. Using bidirectional long short-term memory and conditional random fields for labeling Arabic named entities:A comparative study[C]. Valencia: The Fifth International Conference on Social Networks Analysis,Management and Security,2018:135-140.
[17]	Liu W, Fu X, Zhang Y, et al. Lexicon enhanced Chinese sequence labeling using BERT adapter[C]. Online: Proceedings of the Fifty-ninth Annual Meeting of the Association for Computational Linguistics and the Eleventh International Joint Conference on Natural Language Processing,2021:5847-5858.
[18]	Cui Y, Che W, Liu T, et al. Pre-training with whole word masking for Chinese BERT[J]. IEEE/ACM Transactions on Audio,Speech and Language Processing, 2021, 29(9):3504-3514.
[19]	Xue N. Chinese word segmentation as character tagging[C]. Suzhou: International Journal of Computational Linguistics & Chinese Language Processing,2003:29-48.

Text	Label
振	B-Equipment
动	I- Equipment
盘	I- Equipment
卡	B-Exceptions
胶	I-Exceptions
垫	I-Exceptions
。	O

超参数	数值
BERT-LR	7×10^-5
CRF_LR	2×10^-3
AdamW_LR	1×10^-8
BATCH_SIZE	8
MAX_seq_len	100

模型	评价	设备	故障	操作动作
BERT-CRF	P	0.839 7	0.846 7	0.808 3
	R	0.950 6	0.872 2	0.866 2
	F1	0.891 7	0.859 3	0.836 2
BERT-BiLSTM-CRF	P	0.856 7	0.865 2	0.855 2
	R	0.950 6	0.903 0	0.893 2
	F1	0.901 2	0.879 0	0.873 8
RoBERTa-WWM-BiLSTM-CRF	P	0.850 2	0.869 3	0.817 5
	R	0.969 6	0.921 5	0.912 4
	F1	0.906 0	0.894 6	0.862 4
CEBERT(base)-BiLSTM-CRF	P	0.881 4	0.895 9	0.864 7
	R	0.985 4	0.927 7	0.935 5
	F1	0.930 5	0.911 5	0.898 7
CEBERT-BiLSTM-CRF	P	0.879 5	0.919 1	0.859 9
	R	0.998 1	0.949 2	0.950 9
	F1	0.935 1	0.933 9	0.903 1