威胁情报提取与知识图谱构建技术研究

doi:10.19665/j.issn1001-2400.2023.04.007

西安电子科技大学学报 ›› 2023, Vol. 50 ›› Issue (4): 65-75.doi: 10.19665/j.issn1001-2400.2023.04.007

威胁情报提取与知识图谱构建技术研究

史慧洋^1,²(),魏靖烜³(),蔡兴业³(),王鹤⁴(),高随祥^5,⁶(),张玉清^1,^2,^4,⁶()

1.中国科学院大学计算机科学与技术学院,北京 101408
2.中国科学院大学国家计算机网络入侵防范中心,北京 101408
3.中国科学院大学沈阳计算技术研究所,辽宁沈阳 110168
4.西安电子科技大学网络与信息安全学院,陕西西安 710071
5.中国科学院大学数学科学学院,北京 101408
6.中关村实验室,北京 100094

收稿日期:2023-01-19 出版日期:2023-08-20 发布日期:2023-10-17
通讯作者: 张玉清
作者简介:史慧洋(1988—),女,中国科学院大学博士研究生,E-mail:shihuiyang@ucas.ac.cn;|魏靖烜(1998—),男,中国科学院大学沈阳计算技术研究所博士研究生,E-mail:weijingxuan20@mails.ucas.edu.cn;|蔡兴业(1998—),男,中国科学院大学沈阳计算技术研究所硕士研究生,E-mail:caixingye20@mails.ucas.ac.cn;|王鹤(1998—),男,西安电子科技大学博士研究生,E-mail:hewang@xidian.edu.cn;|高随祥(1962—),男,中国科学院大学博士生导师,E-mail:sxgao@ucas.ac.cn
基金资助:
国家自然科学基金(11991022);国家自然科学基金(12071459)

Research on threat intelligence extraction and knowledge graph construction technology

SHI Huiyang^1,²(),WEI Jingxuan³(),CAI Xingye³(),WANG He⁴(),GAO Suixiang^5,⁶(),ZHANG Yuqing^1,^2,^4,⁶()

1. School of Computer Science and Technology,University of Chinese Academy of Sciences,Beijing 101408,China
2. National Computer Network Intrusion Prevention Center,University of Chinese Academy of Sciences,Beijing 101408,China
3. Shenyang Institute of Computing Technology,University of Chinese Academy of Sciences,Shenyang 110168,China
4. School of Cyber Engineering,Xidian University,Xi’an 710071,China
5. School of Mathematical Sciences,University of Chinese Academy of Sciences,Beijing 101408,China
6. Zhongguancun Laboratory,Beijing 100094,China

Received:2023-01-19 Online:2023-08-20 Published:2023-10-17
Contact: Yuqing ZHANG

摘要/Abstract

摘要：

目前,攻击者使用的基础设施能适应更多的目标环境,成功侵入目标后,使用合法的用户凭证取得信任,并通过不断学习利用新的漏洞达到攻击目的。为了对抗攻击,提高威胁情报的使用价值,提出由情报搜集、信息抽取、本体构建和知识推理构建威胁情报的知识图谱框架,该框架可实现情报中重要指标的搜索和相互关联。然后基于Bert+BiSLTM+CRF 的失陷指标,识别抽取方法,加以正则匹配机制进行输出限制,用于从文本信息中识别抽取失陷指标信息,并进行结构化威胁信息表达标准格式转换。经过横向和纵向对比,该抽取模型在文本信息抽取中的精度和召回率较高。最后,以APT1为例,构建出威胁情报实体关系图,结合对抗战术和技术知识库框架将攻击行为转换为结构化格式,建立本体与原子本体知识图谱;通过知识图谱关联分析数据之间潜在的关联,发现具有相似性和相关性的威胁情报潜在的关联信息和攻击主体,进行威胁情报的关联分析,为制定防御策略提供依据。

关键词: 威胁情报, 神经网络, 本体, 失陷指标抽取, 对抗战术和技术知识库存, 知识图谱

Abstract:

At present,the infrastructure used by attackers can adapt to more target environments.After successfully invading the target,the attackers use legitimate user credentials to gain trust,and continuously learn to exploit new vulnerabilities to achieve the purpose of attacks.In order to combat attacks and to improve the quality and utilization efficiency of the threat intelligence,this paper constructs a knowledge mapping framework of threat intelligence through the following four processes:intelligence collection,information extraction,ontology construction,and knowledge reasoning.The proposed framework can realize the search for and correlation of essential indicators in the intelligence.Then,an indicator of compromise (IOC) recognition extraction method based on the Bert+BISLTM+CRF is proposed and a regular matching mechanism is applied to limit the output for identifying and extracting IOC information from the text information,followed by performing the structured threat information expression (STIX) standard format conversion.The accuracy and recall rate of this extraction model for the text information extraction are higher through horizontal and vertical comparison.Finally,by taking the APT1 as an example,this paper constructs the entity-relationship diagram of threat intelligence.The attack behavior is transformed into a structured format combined with the adversarial tactics,techniques,and common knowledge (ATT & CK) framework.A knowledge map of ontology and atomic ontology is established which is used to analyze the potential associations between data through the knowledge map associations and to discover potential associated information and attack agents in threat intelligence with similarity and correlation.The correlation analysis of threat intelligence is carried out,which provides the basis for the formulation of defense strategy.

Key words: threat intelligence, neural network, ontology, IOC extration, ATT&CK, knowledge graph

中图分类号:

TP399

史慧洋,魏靖烜,蔡兴业,王鹤,高随祥,张玉清. 威胁情报提取与知识图谱构建技术研究[J]. 西安电子科技大学学报, 2023, 50(4): 65-75.

SHI Huiyang,WEI Jingxuan,CAI Xingye,WANG He,GAO Suixiang,ZHANG Yuqing. Research on threat intelligence extraction and knowledge graph construction technology[J]. Journal of Xidian University, 2023, 50(4): 65-75.

图/表 9

图1

表1

图2

图3

图4

表2

表3

图5

图6

参考文献 29

[1]	PADIA A, KALPAKIS K, FERRARO F, et al. Knowledge Graph Fact Prediction via Knowledge-Enriched Tensor Factorization[J]. Journal of Web Semantics, 2019, 59:100497. doi: 10.1016/j.websem.2019.01.004
[2]	GONG S, LEE C. Blocis:Blockchain-Based Cyber Threat Intelligence Sharing Framework for Sybil-Resistance[J]. Electronics, 2020, 9(3):521. doi: 10.3390/electronics9030521
[3]	PUROHIT S, CALYAM P, WANG S, et al. Defensechain:Consortium Blockchain for Cyber Threat Intelligence Sharing and Defense[C]// 2020 2nd Conference on Blockchain Research & Applications for Innovative Networks and Services (BRAINS).Piscataway:IEEE, 2020:112-119.
[4]	TRAN H N, TAKASU A. Analyzing Knowledge Graph Embedding Methods from a Multi-Embedding Interaction Perspective (2019)[J/OL].[2019-03-27]. https://arxiv.org/abs/1903.11406v2.
[5]	何志鹏, 刘鹏, 王鹤. 网络威胁情报标准化建设分析[J]. 信息安全研究, 2021, 7(6):503-511.
	HE Zhipeng, LIU Peng, WANG He. Analysis of the Standardization Construction of Network Threat Intelligence[J]. Information Security Research, 2021, 7(6):503-511.
[6]	孙铭鸿, 蔡蓓蓓. 基于情报、威胁框架等方式追踪溯源方法研究[J]. 江苏通信, 2022, 38(3):109-112.
	SUN Minghong, CAI Beibei. Research on Traceability Methods Based on Intelligence,Threat Framework,and Other Methods[J]. Jiangsu Communication, 2022, 38(3):109-112.
[7]	徐留杰, 翟江涛, 杨康, 等. 一种多源网络安全威胁情报采集与封装技术[J]. 网络安全技术与应用, 2018, 214(10):26-29.
	XU Liujie, ZHAI Jiangtao, YANG Kang, et al. A Multi-Source Network Security Threat Intelligence Collection and Packaging Technology[J]. Network Security Technology and Application, 2018, 214(10):26-29.
[8]	HUANG Z, WEI X, KAI Y. Bidirectional LSTM-CRF Models for Sequence Tagging (2015)[J/OL].[2015-08-09]. https://arxiv.org/abs/1508.01991.
[9]	LONG Z, TAN L, ZHOU S, et al. Collecting Indicators of Compromise from Unstructured Text of Cybersecurity Articles Using Neural-Based Sequence Labelling[C]// 2019 International Joint Conference on Neural Networks (IJCNN).Piscataway:IEEE, 2019:1-8.
[10]	LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural Architectures for Named Entity Recognition(2016)[C/OL].[2016-03-04]. https://arxiv.org/abs/1603.01360v1.
[11]	LANDAUER M, SKOPIK F, WURZENBERGER M, et al. A Framework for Cyber Threat Intelligence Extraction from Raw Log Data[C]// 2019 IEEE International Conference on Big Data (Big Data).Piscataway:IEEE, 2019:3200-3209.
[12]	KUROGOME Y, OTSUKI Y, KAWAKOYA Y, et al. EIGER:Automated IOC Generation for Accurate and Interpretable Endpoint Malware Detection[C]// Proceedings of the 35th Annual Computer Security Applications Conference. New York: ACM, 2019:687-701.
[13]	胡代旺, 焦一源, 李雁妮. 一种新型高效的文库知识图谱实体关系抽取算法[J]. 西安电子科技大学学报, 2021, 48(6):75-83.
	HU Daiwang, JIAO Yiyuan, LI Yanni. A Novel and Efficient Algorithm for Extracting Entity Relationships from Library Knowledge Graph[J]. Journal of Xidian University, 2021, 48(6):75-83.
[14]	郭渊博, 李勇飞, 陈庆礼, 等. 融合Focal Loss的网络威胁情报实体抽取[J]. 通信学报, 2022, 43(7):85-92. doi: 10.11959/j.issn.1000-436x.2022132
	GUO Yuanbo, LI Yongfei, CHEN Qingli, et al. Fusion of Focal Loss for Network Threat Intelligence Entity Extraction[J]. Journal of Communications, 2022, 43(7):85-92. doi: 10.11959/j.issn.1000-436x.2022132
[15]	程顺航, 李志华, 魏涛. 融合自举与语义角色标注的威胁情报实体关系抽取方法[J]. 计算机应用, 2023, 43(5):1445-1453. doi: 10.11772/j.issn.1001-9081.2022040551
	CHENG Shunhang, LI Zhihua, WEI Tao. A Threat Intelligence Entity Relationship Extraction Method Combining Bootstrap and Semantic Role Annotation[J]. Computer Applications, 2023, 43(5):1445-1453. doi: 10.11772/j.issn.1001-9081.2022040551
[16]	石波, 于然, 朱健. 基于知识图谱的网络空间安全威胁感知技术研究[J]. 信息安全研究, 2022, 8(8):845-853.
	SHI Bo, YU Ran, ZHU Jian. Research on Threat Perception Technology for Cyberspace Security Based on Knowledge Graph[J]. Information Security Research, 2022, 8(8):845-853.
[17]	董聪, 姜波, 卢志刚, 等. 面向网络空间安全情报的知识图谱综述[J]. 信息安全学报, 2020, 5(5):56-76. doi: 10.4236/jis.2014.52006
	DONG Cong, JIANG Bo, LU Zhigang, et al. A Survey of Knowledge Map for Cyberspace Security Intelligence[J]. Journal of Information Security, 2020, 5(5):56-76. doi: 10.4236/jis.2014.52006
[18]	WU S, ZHANG Y, CAO W. Network Security Assessment Using a Semantic Reasoning and Graph Based Approach[J]. Computers & Electrical Engineering, 2017, 64:96-109.
[19]	刘强, 祝鹏程. 基于联合学习的端到端威胁情报知识图谱构建方法[J]. 现代计算机, 2021, 16:16-21.
	LIU Qiang, ZHU Pengcheng. A Method for Constructing an End-to-End Threat Intelligence Knowledge Graph Based on Joint Learning[J]. Modern Computer Science, 2021, 16:16-21.
[20]	GONG N Z, LIU B. You are Who You Know and How You Behave:Attribute Inference Attacks via Users' Social Friends and Behaviors[C]// Proceedings of the 25th USENIX Conference on Security Symposium. New York: ACM, 2016:979-995.
[21]	GASCON H, GROBAUER B, SCHRECK T, et al. Mining Attributed Graphs for Threat Intelligence[C]// ACM on Conference on Data Application Security Privacy. New York: ACM, 2017:15-22.
[22]	XU X, CHANG L, QIAN F, et al. Neural Network-Based Graph Embedding for Cross-Platform Binary Code Similarity Detection[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2017:363-376.
[23]	ZHANG K, LIU J. Review on the Application of Knowledge Graph in Cyber Security Assessment[J]. IOP Conference Series:Materials Science and Engineering, 2020, 768(5):052103. doi: 10.1088/1757-899X/768/5/052103
[24]	LIU L, TSAI W T, BHUIYAN M Z A, et al. Automatic Blockchain Whitepapers Analysis via Heterogeneous Graph Neural Network[J]. Journal of Parallel and Distributed Computing, 2020, 145:1-12. doi: 10.1016/j.jpdc.2020.05.014
[25]	OUYANG S, DONG D, XU Y, et al. Communication Optimization Strategies for Distributed Deep Neural Network Training:A Survey[J]. Journal of Parallel and Distributed Computing, 2021, 149:52-65. doi: 10.1016/j.jpdc.2020.11.005
[26]	MAVROEIDIS V, BROMANDER S. Cyber Threat Intelligence Model:An Evaluation of Taxonomies,Sharing Standards,and Ontologies within Cyber Threat Intelligence[C]// 2017 European Intelligence and Security Informatics Conference (EISIC).Piscataway:IEEE, 2017:91-98.
[27]	MOUBARAK J, BASSIL C, ANTOUN J. On the Dissemination of Cyber Threat Intelligence Through Hyperledger[C]// 2021 17th International Conference on the Design of Reliable Communication Networks (DRCN).Piscataway:IEEE, 2021:1-6.
[28]	崔琳, 杨黎斌, 何清林, 等. 基于开源信息平台的威胁情报挖掘综述[J]. 信息安全学报, 2022, 7(1):1-26. doi: 10.4236/jis.2016.71001
	CUI Lin, YANG Libin, HE Qinglin, et al. A Review of Threat Intelligence Mining Based on Open Source Information Platform[J]. Journal of Information Security, 2022, 7(1):1-26. doi: 10.4236/jis.2016.71001
[29]	左开中, 刘蕊, 赵俊, 等. 融合语义信息的时空关联位置隐私保护方法[J]. 西安电子科技大学学报, 2022, 49(1):67-77.
	ZUO Kaizhong, LIU Rui, ZHAO Jun, et al. Method for the Protection of Spatiotemporal Correlation Location Privacy with Semantic Information[J]. Journal of XidIan University, 2022, 49(1):67-77.

序号	标题	作者	发布时间	阅读量	评论量	正文内容
1	蔓灵花APT行动攻击报告	360安全	2016/11/15	53 552	12	移动平台攻击增加,跨平台攻击渐成趋势。本次捕获的蔓灵花攻击行动中,不仅有针对Windows目标的攻击,还有针对移动Android系统的攻击,黑客通过假冒应用侵入目标的移动设备,上传用户信息,并监控用户操作
2	全球高级持续性威胁(APT) 2019年报告	奇安信威胁雷达	2020/03/02	167 331	5	过去的一年,在网络威胁(Cyber Threat)领域度过了颇为不平静的一年。网络威胁和攻击似乎更为广泛地应用于地缘政治和军事冲突之下等

模型	精确率	召回率	F1
正则	30.4	34.5	32.3
BiLSTM	65.1	52.7	58.2
BiLSTM+CRF	70.4	75.0	72.6
Bert+BiLSTM+CRF	76.3	75.8	76.0
Bert+BiLSTM+CRF+正则	78.2	77.0	77.6

模型	精确率	召回率	F1
Baseline^[7]	60.1	42.7	49.9
Huang et al.^[8]	60.2	35.0	44.3
Lample^[10]	78.3	74.8	76.5
文中模型	78.2	77.0	77.6

威胁情报提取与知识图谱构建技术研究

Research on threat intelligence extraction and knowledge graph construction technology

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 29

相关文章 15

Metrics

本文评价

推荐阅读 10

[1]	范文同,李震宇,张涛,罗向阳. 基于隐写噪声深度提取的JPEG图像隐写分析[J]. 西安电子科技大学学报, 2023, 50(4): 157-169.
[2]	董勐,高一鸣,潘伟涛,邱智亮,杨建磊,邸志雄,郑凌. RELIC-GNN:一种高效的状态寄存器识别算法[J]. 西安电子科技大学学报, 2023, 50(3): 142-150.
[3]	蔡国庆,刘玲,张冲,周一青. 基于GNN-LSTM-CNN网络的6G车辆轨迹预测算法[J]. 西安电子科技大学学报, 2023, 50(3): 50-60.
[4]	陈永,牛凯玉,康婕. LSTM循环神经网络的高速铁路越区切换算法[J]. 西安电子科技大学学报, 2023, 50(1): 76-84.
[5]	张泽欢, 刘强, 国狄非. 面向大规模零样本图像识别的高效算法框架[J]. 西安电子科技大学学报, 2022, 49(6): 103-110.
[6]	任佳兴, 曹玉东, 曹睿, 闫佳. 一种采用动态子空间的小样本图像分类算法[J]. 西安电子科技大学学报, 2022, 49(5): 166-174.
[7]	刘侍刚,张同,杨建功,葛宝. 递进式空洞残差深度双目立体匹配网络[J]. 西安电子科技大学学报, 2022, 49(5): 175-180.
[8]	王侃, 王孟洋, 刘鑫, 田国强, 李川, 刘伟. 融合自注意力机制与CNN-BiGRU的事件检测[J]. 西安电子科技大学学报, 2022, 49(5): 181-188.
[9]	马仑,刘鑫,赵斌,王瑞平,廖桂生,张亚静. 利用多头-连体神经网络实现障碍行为识别[J]. 西安电子科技大学学报, 2022, 49(4): 100-108.
[10]	井佩光,李亚鑫,苏育挺. 一种多模态特征编码的短视频多标签分类方法[J]. 西安电子科技大学学报, 2022, 49(4): 109-117.
[11]	蒋芳,黄兴,胡梦钰,王翊,许耀华,胡艳军. 降噪自编码器辅助的下行MIMO-SCMA编解码方法[J]. 西安电子科技大学学报, 2022, 49(3): 74-82.
[12]	史加荣,李金红. 新型深度矩阵分解及其在推荐系统中的应用[J]. 西安电子科技大学学报, 2022, 49(3): 171-182.
[13]	时云龙,袁文浩,胡少东,娄迎曦. 一种用于实时语音增强的卷积准循环网络[J]. 西安电子科技大学学报, 2022, 49(3): 183-190.
[14]	周鹏,杨军. 索引边缘几何卷积神经网络用于点云分类[J]. 西安电子科技大学学报, 2022, 49(2): 207-217.
[15]	张敏,贾海蓉,张刚敏,王素英. 结合自适应软掩模和混合特征的语音增强[J]. 西安电子科技大学学报, 2022, 49(2): 108-115.