一种利用注意力增强卷积的暗网用户对齐方法

doi:10.19665/j.issn1001-2400.2023.04.020

摘要/Abstract

摘要：

暗网用户在地下市场从事大量违法犯罪活动,暗网的匿名性给暗网用户之间的沟通交流带来了极大的便利,但也给执法人员带来了极大困难。近年来,深度神经网络在各个领域取得广泛成功,越来越多的研究者开始利用神经网络对匿名的网络文本作者进行身份识别。为了更好地进行暗网用户对齐,寻找更多同一身份的不同用户,笔者借用神经网络方法进行暗网用户身份识别和对齐。然而已有的方法主要面向短文本,不擅长处理全局和长序列信息。文中提出了一种自注意机制来增强卷积算子,利用长序列信息来建模暗网用户发表的网络文本的方法,从文本内容入手,对匿名的暗网用户进行多账号关联,达到聚合多个匿名账号信息的目的,为获取用户的真实身份提供更多线索。笔者在两个不同的暗网市场论坛进行全面评估,将提出的方法与当前最先进的技术进行了比较。结果表明提出的方法非常有效,在两个公开数据集上平均检索排名(MRR)分别提高约2.9%和3.6%,Recall@10分别提高约2.3%和3.0%。这项评估为该方法在暗网市场论坛中的有效性提供了强有力的证据。

关键词: 文本嵌入, 注意力机制, 卷积算子, 长序列信息

Abstract:

Dark network users engage in a large number of illegal and criminal activities in the underground market.The anonymity of the dark network brings great convenience to the communication between users of the dark network,but great difficulties to the police.In recent years,the deep neural network has been widely successful in various fields,and more and more researchers have begun to use the neural network to identify anonymous network text authors.In order to better align users in the dark web and find more different users with the same identity,we use the neural network method to identify and align users in the dark web.However,the existing methods focus mainly on the short text and are not good at dealing with the global and long sequence information.In this paper,we propose a self-attention mechanism to enhance the convolution operator and use long sequence information to strengthen the user representation,named DACN.DACN starts from the text content,and multiple account associations are carried out for anonymous dark web users to aggregate information from multiple anonymous accounts,proving mores clues for obtaining the users’true identity.Our recent analysis involves conducting a thorough assessment of two distinct dark web market forums,whereby we evaluate our methodology in comparison to the current state-of-the-art techniques.Experimental results show that our approach is remarkably effective,with a demonstrated average mean retrieval ranking (MRR) enhancement of 2.9% and 3.6%,as well as an improved Recall@10 of 2.3% and 3.0%.This evaluation offers robust evidence of the efficacy of our approach in dark web market forums.

Key words: text embedding, attention mechanism, convolutional networks, long sequence information

中图分类号:

TP18

杨燕燕,杜彦辉,刘洪梦,赵佳鹏,时金桥,王学宾. 一种利用注意力增强卷积的暗网用户对齐方法[J]. 西安电子科技大学学报, 2023, 50(4): 206-214.

YANG Yanyan,DU Yanhui,LIU Hongmeng,ZHAO Jiapeng,SHI Jinqiao,WANG Xuebin. Dark web author alignment based on attention augmented convolutional networks[J]. Journal of Xidian University, 2023, 50(4): 206-214.

图/表 7

图1

图2

图3

表1

表2

图4

图5

参考文献 27

[1]	KIM Y. Convolutional Neural Networks for Sentence Classification (2014)[C/OL].[2014-08-25]. https://arxiv.org/abs/1408.5882v2.
[2]	ANDREWS M, WITTEVEEN S. Unsupervised Natural Question Answering with a Small Model (2019)[C/OL].[2019-11-19]. https://arxiv.org/abs/1911.08340v1.
[3]	MANERIKERP, HE Y, PARTHASARATHY S. SYSML:StYlometry with Structure and Multitask Learning:Implications for Darknet Forum Migrant Analysis (2021)[J/OL].[2021-04-01]. https://arxiv.org/abs/2104.00764.
[4]	BELLO I, ZOPH B, LE Q, et al. Attention Augmented Convolutional Networks[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV).Piscataway:IEEE, 2019:3285-3294.
[5]	STANOVSKY G, GRUHL D, MENDES P. Recognizing Mentions of Adverse Drug Reaction in Social Media Using Knowledge-Infused Recurrent Models[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics.Stroudsburg:ACL, 2017:142-151.
[6]	SEBASTIAN R, PARSA G, JOHN G B. Character-Level and Multi-Channel Convolutional Neural Networks for Large-Scale Authorship Attribution (2016)[J/OL].[2016-09-21]. https://arxiv.org/abs/1609.06686.
[7]	PRASHA S, SEBASTIAN S, FABIO G, et al. Convolutional Neural Networks for Authorship Attribution of Short Texts[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics.Stroudsburg:ACL, 2017:669-674.
[8]	RICO S, BARRY H, ALEXANDRA B. Neural Machine Translation of Rare Words with Subword Units[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL, 2016:1715-1725.
[9]	PIOTR G, EWA J, MACIEJ P. Sparse Coding in Authorship Attribution for Polish Tweets[C]// Proceedings of the International Conference on Recent Advances in Natural Language Processing.Stroudsburg:ACL, 2019:409-417.
[10]	MOHAMMADREZA E, MIHAI S, SAGAR S, et al. Detecting Cyber Threats in Non-English Dark Net Markets:A Crosslingual Transfer Learning Approach[C]// 2018 IEEE International Conference on Intelligence and Security Informatics (ISI).Piscataway:IEEE, 2018:85-90.
[11]	ASHISH V, NOAM S, NIKI P, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017:6000-6010.
[12]	DZMITRY B, KYUNGHYUN C, YOSHUA B. Neural Machine Translation by Jointly Learning to Align and Translate (2014)[C/OL].[2014-09-01]. https://arxiv.org/abs/1409.0473.
[13]	AADAMS W Y, DAVID D, MINH-THANG L, et al. QAnet:Combining Local Convolution with Global Self-Attention for Reading Comprehension (2018)[C/OL].[2018-04-23]. https://arxiv.org/abs/1804.09541.
[14]	DAVID R.S, CHEN L, QUOC V. The Evolved Transformer (2019)[C/OL].[2019-01-30]. https://arxiv.org/abs/1901.11117.
[15]	JIE H, LI S, SAMUEL A, et al. Gather-Excite:Exploiting Feature Context in Convolutional Neural Networks[C]// Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York: ACM, 2018:9423-9433.
[16]	JIE H, LI S, GANG S. Squeeze-and-Excitation Networks[C]// Proceedings of the IEEE conference on computer vision and pattern recognition.Piscataway:IEEE, 2018:7132-7141.
[17]	JONGCHAN P, SANGHYUN W, JOON-YOUNG L, et al. Bam:Bottleneck Attention Module (2018)[C/OL].[2018-07-17]. https://arxiv.org/abs/1807.06514v1.
[18]	SANGHYUN W, JONGCHAN P, JOON-YOUNG L, et al. Cbam:Convolutional Block Attention Module[C]// Proceedings of the European Conference on Computer Vision (ECCV).Heidelberg:Springer, 2018:3-19.
[19]	WANG X L, ROSS G, ABHINAV G, et al. Non-Local Neural Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:7794-7803.
[20]	FAN Y, ZHANG Y, YE Y, et al. AutomaticOpioid User Detection from Twitter:Transductive Ensemble Built on Different Metagraph Based Similarities over Heterogeneous Information Network[C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. New York: ACM, 2018:3357-3363.
[21]	HOU S, YE Y, SONG Y, et al. Hindroid:AnIntelligent Android Malware Detection System Based on Structured Heterogeneous Information network[C]// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2017:1507-1515.
[22]	FU T, LEE W, LEI Z. 2017.Hin2vec:ExploreMeta-Paths in Heterogeneous Information Networks for Representation Learning[C]// Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. New York: ACM, 2017:1797-1806.
[23]	DONG Y, NITESH V.C, ANANTHRAM S. Metapath2vec:Scalable Representation Learning for heterogeneous Networks[C]// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mning. New York: ACM, 2017:135-144.
[24]	ZHANG Y, FAN Y, SONG W, et al. Your Style Your Identity:Leveraging Writing and Photography Styles for Drug Trafficker Identification in Darknet Markets over Attributed Heterogeneous Information Network[C]// International World Wide Web Conference Committee. New York: ACM, 2019:3448-3454.
[25]	RAMNATH K, SHWETA Y, RAMINTA D, et al. Edarkfind:Unsupervised Multi-View Learning for Sybil Account Detection[C]// International World Wide Web Conference Committee. New York: ACM, 2020:1955-1965.
[26]	NICHOLAS A, MARCUS B. LearningInvariant Representations of Social Media Users (2019)[C/OL].[2019-10-11]. https://arxiv.org/abs/1910.04979.
[27]	JACOB D, CHANG M, KENTON L, et al. BERT:Pre-Training of Deep Bidirectional Transformers for Language Understanding (2018)[C/OL].[2018-10-11]. https://arxiv.org/abs/1810.04805.

数据集名称	训练集帖子数量	测试集帖子数量	训练集用户数量	测试集用户数量
丝绸之路	379 382	381 959	6 585	8 865
广场市场	175 978	179 482	3 115	4 209

方法	广场市场		丝绸之路
方法	MRR	Recall@10	MRR	Recall@10
只考虑文本建模
TextCNN(2014)^[1]	0.126	0.214	0.036	0.073
Transformer (2017)^[11]	0.091	0.133	0.026	0.043
DACN	0.131	0.179	0.080	0.124
考虑用户建模
IUR (2019)^[2]	0.114	0.218	0.109	0.190
SYSML-CNN (2021)^[3]	0.152	0.279	0.157	0.252
SYSML-Transformer (2021)^[3]	0.133	0.235	0.103	0.175
DACN	0.181	0.302	0.193	0.282