基于字符增强的工业设备故障命名实体识别

doi:10.16180/j.cnki.issn1007-7820.2024.10.007

Abstract

Abstract:

To address the issues of sparse training data, complex entity structures, and uneven entity distribution in the industrial equipment failure domain, this study constructs an industrial equipment failure named entity recognition corpus. Due to the difficulty of character-level named entity recognition models in representing the professional vocabulary information in the field of industrial equipment failure, this study proposes a character-enhanced industrial equipment failure named entity recognition model to address this problem. In the embedding layer, professional vocabulary information is directly fused between the Transformer layers of RoBERTa-WWM (Robustly Optimized BERT Pretraining Approach with Whole Word Masking) to allocate word information to each of its constituent characters for enhanced semantics. The global semantic information is obtained through a BiLSTM(Bidirectional Long Short-Term Memory), and the CRF(Conditional Random Field) is used to learn the dependency relationship between adjacent labels to obtain the optimal sentence-level label sequence. Experimental results demonstrate that the proposed model has good performance on industrial equipment fault named entity recognition tasks, with an average F1 score of 92.403%.

Key words: industrial equipment failure, corpus, named entity recognition, RoBERTa-WWM, professional word embedding, BiLSTM, CRF, deep learning

CLC Number:

TP183

ZHANG Yang, LIU Jin. Character Enhancement Based on Named Entity Recognition for Industrial Equipment Faults[J].Electronic Science and Technology, 2024, 37(10): 48-54.

Figures/Tables 10

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Table 1.

Table 2.

Table 3.

Figure 6.

Figure 7.

References 19

[1]	王骏东, 杨军, 裴洋舟, 等. 基于知识图谱的配电网故障辅助决策研究[J]. 电网技术, 2021, 45(6):2101-2112.
	Wang Jundong, Yang Jun, Pei Yangzhou, et al. Distribution network fault assistant decision-making based on knowledge graph[J]. Power System Technology, 2021, 45(6):2101-2112.
[2]	Martinez-Gil J, Buchgeher G, Gabauer D, et al. Root cause analysis in the industrial domain using knowledge graphs:A case study on power transformers[J]. Procedia Computer Science, 2022, 20(8):944-953.
[3]	Zhang Y, Kang J, Dai W. Non-functional requirements e-licitation based on domain knowledge graph for automatic code generation of industrial cyber-physical systems[C]. Toronto: IECON the Forty-seventh Annual Conference of the IEEE Industrial Electronics Society,2021:1-6.
[4]	Chen X, Ouyang C, Liu Y, et al. Improving the named e-ntity recognition of Chinese electronic medical records by combining domain dictionary and rules[J]. International Journal of Environmental Research and Public Health, 2020, 17(8):2687-2703.
[5]	Feng X, Li Y, Hang Z, et al. TBR-NER:Research on CO-VID-19 text information extraction based on joint learning of topic recognition and named entity recognition[J]. Journal of Sensors, 2022, 20(22):1-15.
[6]	杨鹤, 于红, 刘巨升, 等. 基于BERT+BiLSTM+CRF深度学习模型和多元组合数据增广的渔业标准命名实体识别[J]. 大连海洋大学学报, 2021, 36(4):661-669.
	Yang He, Yu Hong, Liu Jusheng, et al. Fishery standard named entity recognition based on BERT+BiLSTM+C-RF deep learning model and multivariate combined data augmentation[J]. Journal of Dalian Fisheries University, 2021, 36(4):661-669.
[7]	Ćeović H, Kurdija A S, Delać G, et al. Named entity recognition for addresses:An empirical study[J]. IEEE Access, 2022(10):42108-42120.
[8]	孟昕. 基于深度学习的法律文书识别方法研究[J]. 电子科技, 2019, 32(12):84-86.
	Meng Xin. Research on recognition method of legal documents based on deep learning[J]. Electronic Science and Technology, 2019, 32(12):84-86.
[9]	Zhang H, Wang X, Liu J, et al. Chinese named entity recognition method for the finance domain based on enhanced features and pretrained language models[J]. Information Sciences, 2023, 62(5):385-400.
[10]	Brandsen A, Verberne S, Lambers K, et al. Can BERT dig it?Named entity recognition for information retrieval in the archaeology domain[J]. Journal on Computing and Cultural Heritage, 2022, 15(3):1-18.
[11]	Zheng K, Sun L, Wang X, et al. Named entity recognition in electric power metering domain based on attention mechanism[J]. IEEE Access, 2021(9):152564-152573.
[12]	Ivanin V, Artemova E, Batura T, et al. RuREBus:A case study of joint named entity recognition and relation extraction from E-government domain[C]. Skolkovo:Analysis of Images, Social Networks and Texts: The Ninth International Conference,2020:19-27.
[13]	Li J, Sun A, Han J, et al. A survey on deep learning for named entity recognition[J]. IEEE Transactions on Knowledge and Data Engineering, 2020, 34(1):50-70.
[14]	Devlin J, Chang M, Lee K, et al. BERT:Pretraining of deep bidirectional transformers for language understanding[C]. Minneapolis:Proceedings of the Seventeenth Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2019:4171-4186.
[15]	Hochreiter S, Schmidhuber J. Long shortterm memory[J]. Neural Computation, 1997, 9(8):1735-1780. doi: 10.1162/neco.1997.9.8.1735 pmid: 9377276
[16]	Sa'a D A A, Tawalbeh S K, Al-Smadi M, et al. Using bidirectional long short-term memory and conditional random fields for labeling Arabic named entities:A comparative study[C]. Valencia: The Fifth International Conference on Social Networks Analysis,Management and Security,2018:135-140.
[17]	Liu W, Fu X, Zhang Y, et al. Lexicon enhanced Chinese sequence labeling using BERT adapter[C]. Online: Proceedings of the Fifty-ninth Annual Meeting of the Association for Computational Linguistics and the Eleventh International Joint Conference on Natural Language Processing,2021:5847-5858.
[18]	Cui Y, Che W, Liu T, et al. Pre-training with whole word masking for Chinese BERT[J]. IEEE/ACM Transactions on Audio,Speech and Language Processing, 2021, 29(9):3504-3514.
[19]	Xue N. Chinese word segmentation as character tagging[C]. Suzhou: International Journal of Computational Linguistics & Chinese Language Processing,2003:29-48.

Text	Label
振	B-Equipment
动	I- Equipment
盘	I- Equipment
卡	B-Exceptions
胶	I-Exceptions
垫	I-Exceptions
。	O

超参数	数值
BERT-LR	7×10^-5
CRF_LR	2×10^-3
AdamW_LR	1×10^-8
BATCH_SIZE	8
MAX_seq_len	100

模型	评价	设备	故障	操作动作
BERT-CRF	P	0.839 7	0.846 7	0.808 3
	R	0.950 6	0.872 2	0.866 2
	F1	0.891 7	0.859 3	0.836 2
BERT-BiLSTM-CRF	P	0.856 7	0.865 2	0.855 2
	R	0.950 6	0.903 0	0.893 2
	F1	0.901 2	0.879 0	0.873 8
RoBERTa-WWM-BiLSTM-CRF	P	0.850 2	0.869 3	0.817 5
	R	0.969 6	0.921 5	0.912 4
	F1	0.906 0	0.894 6	0.862 4
CEBERT(base)-BiLSTM-CRF	P	0.881 4	0.895 9	0.864 7
	R	0.985 4	0.927 7	0.935 5
	F1	0.930 5	0.911 5	0.898 7
CEBERT-BiLSTM-CRF	P	0.879 5	0.919 1	0.859 9
	R	0.998 1	0.949 2	0.950 9
	F1	0.935 1	0.933 9	0.903 1

Character Enhancement Based on Named Entity Recognition for Industrial Equipment Faults

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 19

Related Articles 15

Metrics

Comments

Recommended 0

[1]	HE Zhiqiang, SUN Zhanquan. Swin-Transformer-Based Carotid Ultrasound Image Plaque Segmentation [J]. Electronic Science and Technology, 2024, 37(9): 48-56.
[2]	HE Xing, HUANG Yongming, ZHU Yong. Pavement Pothole Detection Method Based on Improved YOLOv5 [J]. Electronic Science and Technology, 2024, 37(7): 53-59.
[3]	YE Yuxin, JU Zhiyong, LAI Ying. Traffic Sign Detection Algorithm Incorporating Receptive Field Enhancement Module and Attention Mechanism [J]. Electronic Science and Technology, 2024, 37(6): 8-16.
[4]	YUAN Ao, QI Jinpeng, JIA Can, XUE Yuxin, GUO Yangyang. A Classification Method of Time Series Pathological Data Segments Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(6): 84-91.
[5]	ZHU Zihao, SONG Yan. Lightweight Capsule Network Fusing Attention and Capsule Pooling [J]. Electronic Science and Technology, 2024, 37(5): 1-8.
[6]	XIA Rongcheng, LIU Deer. Vehicle Detection and Analysis in Urban Waterlogging Area Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(5): 18-24.
[7]	GAO Dian, ZHANG Jing. Short-Term Load Forecasting Based on CEEMD-ITSA-BiLSTM Combined Model [J]. Electronic Science and Technology, 2024, 37(4): 30-37.
[8]	MA Wenjie, ZHANG Xuanxiong. Research on Blind Roads and Obstacle Recognition Algorithm Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(3): 75-83.
[9]	JU Ping, SONG Yan, ZHANG Yingjie, XU Yifu, SHAO Hang. Research on Agricultural Crop Diseases and Pests Classification Based on Masked Autoencoding [J]. Electronic Science and Technology, 2024, 37(10): 23-29.
[10]	FENG Ting, YING Jie, YANG Haima, LI Fang. Detection of Cervical Lesions Based on Multi-Scale Features and Attention Mechanism [J]. Electronic Science and Technology, 2024, 37(10): 30-39.
[11]	YU Zhihong,LI Feifei. Semi-Supervised Medical Image Segmentation Method Based on Meta-Learning and Neural Architecture Search [J]. Electronic Science and Technology, 2024, 37(1): 17-23.
[12]	HUANG Zixuan,LI Qiaoxing. Research on Identification of Urban Illegal Vehicles Based on Random Forest Model [J]. Electronic Science and Technology, 2024, 37(1): 66-71.
[13]	LI Zenghui,WANG Wei. Research Progress of Medical Image Segmentation Method Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(1): 72-80.
[14]	WANG Jiawei,YU Xiao. Review of Text Classification Research Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(1): 81-86.
[15]	Bin ,WANG Sen. Visual Detection of Structural Cracks Using Depth Deformable Contour ModelLAI [J]. Electronic Science and Technology, 2023, 36(9): 35-40.