融入注意力网络的深度分解机推荐算法

doi:10.16180/j.cnki.issn1007-7820.2023.01.006

Abstract

Abstract:

Recommender system can find information that satisfies the user's individual needs from huge amount of information. With the development of deep learning, deep learning has been widely applied in recommender systems. CTR prediction plays an important role in recommender system and has been widely used in many fields such as personalized recommendation, information retrieval, online advertising and so on. For the issue of large and sparse data in recommender system, this study merges xDeepFM model with attention network, and proposes a new CTR prediction model based on deep learning, which is called Atte-xDeepFM model. This model can solve the issue of feature scarcity, effectively learn the interactions relationship between features, and does not need to manually extract useful information in feature engineering. The comparative experiments on Avazu and Criteo data sets prove the effectiveness of the proposed model. Compared with the algorithm model commonly used in CTR prediction, the proposed model has better recommendation effect.

Key words: recommender system, deep learning, personalized recommendation, computational advertising, CTR prediction, factorization machine, attention network, feature sparsity

CLC Number:

TP393

WU Tong,YU Lianzhi. The Recommendation Algorithm of Extreme Deep Factorization Machine Merged with Attention Network[J].Electronic Science and Technology, 2023, 36(1): 38-43.

Figures/Tables 4

References 29

[1]	项亮. 推荐系统实践[M]. 北京: 人民邮电出版社, 2012.
	Xiang Liang. Practice of recommender system[M]. Beijing: The People's Posts and Telecommunications Press, 2012.
[2]	Pavan K, Vairachilai S, Potluri S, et al. Recommender systems: Algorithms and applications[M]. Boca Raton: CRC Press, 2021.
[3]	Hsu C L. A multi-valued and sequential-labeled decision tree method for recommending sequential patterns in cold-start situations[J]. Applied Intelligence, 2021, 51(1):506-526. doi: 10.1007/s10489-020-01806-0
[4]	Wang Z, Yu Q, Shen C, et al. Feature selection in click-through rate prediction based on gradient boosting[M].Berlin:International Conference on Intelligent Data Engineering and Automated Learning, 2016.
[5]	Wen H, Zhang J, Lin Q, et al. Multi-level deep cascade trees for conversion rate prediction in recommendation System[C]. Honolulu: Proceedings of the Thirty-third AAAI Conference on Artificial Intelligence, 2019.
[6]	Ling X, Deng W, Gu C, et al. Model ensemble for click prediction in bing search ads[C]. Perth: Proceedings of the Twenty-sixth International Conference on World Wide Web Companion, 2019.
[7]	McMahan H B, Holt G, Scully D, et al. Ad click prediction a view from the trenches[C]. Chicago: Proceedings of the Nineteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2013.
[8]	Ta A P. Factorization machines with follow- the- regularized- leader for CTR prediction in display advertising[C]. Santa Clara: Proceedings of the IEEE International Conference on Big Data, 2015.
[9]	Zhang S, Yao L, Sun A, et al. Deep learning based recommender system:A survey and new perspectives[C]. Como: Proceedings of the Eleventh ACM Conference on Recommender Systems, 2017.
[10]	Cheng H T, Koc L, Harmsen J, et al. Wide & Deep learning for recommender systems[C]. Boston: Proceedings of the First Workshop on Deep Learning for Recommender Systems, 2016.
[11]	Guo H, Tang R, Ye Y, et al. DeepFM: A factorization-machine based neural network for CTR prediction[C]. Melbourne: Proceedings of the Twenty-sixth International Joint Conference on Artifcial Intelligence, 2017.
[12]	Strub F, Gaudel R, Mary J. Hybrid recommender system based on autoencoders[C]. Boston: Proceedings of the First Workshop on Deep Learning for Recommender Systems, 2016.
[13]	Tahmasebi H, Ravanmehr R, Mohamadrezaei R. Social movie recommender system based on deep autoencoder network using Twitter data[J]. Neural Computing and Applications, 2021, 33(5):1607-1623. doi: 10.1007/s00521-020-05085-1
[14]	Gong Y, Zhang Q. Hashtag recommendation using Attention-based convolutional neural network[C]. New York: Proceedings of the Conference Division and the AI Journal Division, 2016.
[15]	Suglia A, Greco C, Musto C, et al. A deep architecture for content-based recommendations exploiting recurrent neural networks[C]. Bratislava: Proceedings of the Twenty-fifth Conference on User Modeling, Adaptation and Personalization, 2017.
[16]	许凤翔. 一种改进相似度的协同过滤算法实现[J]. 电子科技, 2020, 33(2):54-59.
	Xu Fengxiang. Implementation of a collaborative filtering algorithm based on improved similarity[J]. Electronic Science and Technology, 2020, 33(2):54-59.
[17]	卢佳伟, 陈玮, 尹钟. 融合TextRank算法的中文短文本相似度计算[J]. 电子科技, 2020, 33(10):51-56.
	Lu Jiawei, Chen Wei, Yin Zhong. Chinese short text similarity calculation based on TextRank algorithm[J]. Electronic Science and Technology, 2020, 33(10):51-56.
[18]	Gao L, Dai K, Gao L, et al. Expert knowledge recommendation systems based on conceptual similarity and space mapping[J]. Expert Systems with Applications, 2019, 13(6):242-251.
[19]	Behera D K, Da S M, Swetanisha S. Predicting users' preferences for movie recommender system using rstricted boltzmann machine[M]. Singapore: Springer Nature, 2019.
[20]	Rendle S. Factorization machines[C]. Sydney: Proceedings of the Tenth IEEE International Conference on Data Mining, 2010.
[21]	Juan Y, Zhuang Y, Chin W S, et al. Field-aware factorization machines for CTR prediction[C]. Boston: Proceedings of the Tenth ACM Conference on Recommender System, 2016.
[22]	Blondel M, Fujino A, Ueda N, et al. Higher-order factorization machines[C]. New York: Proceedings of the Advances in Neural Information Processing Systems, 2016.
[23]	Xiao J, Ye H, He X, et al. Attentional factorization machines:Learning the weight of feature interactions via attention networks[C]. Melbourne: Proceedings of the Twenty-sixth International Joint Conference on Artificial Intelligence, 2017.
[24]	Zhang W, Du T, Wang J. Deep learning over multi-field categorical data[C]. Springer: Proceedings of the European Conference on Information Retrieval, 2016.
[25]	Qu Y, Han C, Kan R, et al. Product-based neural networks for user response prediction[C]. Barcelona: Proceedings of the IEEE International Conference on Data Mining, 2016.
[26]	Wang R, Fu B, Fu G, et al. Deep & Cross network for Ad click predictions[C]. Halifax: Proceedings of the Twenty-third ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2017.
[27]	He X, Chua T S. Neural factorization machines for sparse predictive analytics[J]. Proceeding of the Fortieth International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017.
[28]	Lian J, Zhou X, Zhang F, et al. xDeepFM:Combining explicit and implicit feature interactions for recommender systems[C]. London: Proceedings of the Twenty-fourth ACM SIGKDD International Conference, 2018.
[29]	Hu J, Shen L, Sun G, et al. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.

数据集	样本量	特征数(稀疏化)	所占空间
Avazu	41 MB	14 kB	5.6 GB
Criteo	45 MB	45 kB	15.2 GB

模型名称	Avazu		Criteo
模型名称	AUC	Logloss	AUC	Logloss
AFM	0.742 097	0.354 247	0.736 624	0.492 851
NFM	0.741 831	0.354 501	0.736 438	0.493 016
CIN	0.744 281	0.359 772	0.743 343	0.489 487
Atte-CIN	0.746 040	0.356 845	0.744 328	0.486 356
DeepFM	0.762 503	0.334 862	0.783 259	0.458 730
xDeepFM	0.761 042	0.341 024	0.771 216	0.463 894
Atte-xDeepFM	0.764 892	0.315 846	0.788 733	0.448 124

The Recommendation Algorithm of Extreme Deep Factorization Machine Merged with Attention Network

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 4

References 29

Related Articles 15

Metrics

Comments

Recommended 10

[1]	YU Qiongfang,NIU Dongyang. Mixed Prediction of Mine Pressure Time and Space Based on LSTM Network [J]. Electronic Science and Technology, 2023, 36(2): 67-72.
[2]	ZUO Bin,LI Feifei. An Effective Segmentation Method for COVID-19 CT Image Based on Attention Mechanism and Inf-Net [J]. Electronic Science and Technology, 2023, 36(2): 22-28.
[3]	ZHAO Wenjun,ZHAI Han,ZHANG Hongyan. Total Variation and Sparsity Regularized Deep Nonnegative Matrix Factorization for Hyperspectral Unmixing [J]. Electronic Science and Technology, 2023, 36(2): 53-60.
[4]	BI Jiazhen,SHEN Tuo,ZHANG Xuanxiong. A Research on Distance Measurement Between Trains in Rail Transit Based on Machine Vision [J]. Electronic Science and Technology, 2022, 35(9): 37-43.
[5]	ZHANG Qiaomu,ZHONG Qianwen,SUN Ming,LUO Wencheng,CHAI Xiaodong. Research on Dynamic Monitoring Method of Pantograph-Net Contact Position in Complex Environment [J]. Electronic Science and Technology, 2022, 35(8): 66-72.
[6]	ZHAO Xuan,ZHOU Fan,YU Hancheng. Improved YOLOv3 Model Based on New Feature Extraction and Fusion Module [J]. Electronic Science and Technology, 2022, 35(7): 40-45.
[7]	Peng CHEN,Zilong LIU. Arrhythmia Recognition Based on GAN-CNN [J]. Electronic Science and Technology, 2022, 35(3): 45-50.
[8]	DENG Lei,SUN Peiyang. Research on Network Public Opinion Monitoring System Based on Deep Learning [J]. Electronic Science and Technology, 2022, 35(12): 97-102.
[9]	YU Runrun,JIANG Xiaoyan,ZHU Kaiying,JIANG Guanghao. Real-Time Image Semantic Segmentation Based on Contextual Attention Mechanism [J]. Electronic Science and Technology, 2022, 35(12): 57-63.
[10]	ZHANG Ye,BAO Liang. Research Progress of Service Composition Based on Machine Learning [J]. Electronic Science and Technology, 2022, 35(11): 58-63.
[11]	ZHANG Wei,LIU Na,JIANG Yang,LI Qingdu. Garbage Detection and Classification Based on YOLO Neural Network [J]. Electronic Science and Technology, 2022, 35(10): 45-50.
[12]	CHENG Xiaoya,ZHANG Lei. Research on Occluded Face Recognition Method Based on Deep Learning [J]. Electronic Science and Technology, 2022, 35(1): 35-39.
[13]	ZHAO Chong,CHI Mengmeng,CHU Cong,ZHANG Peng. Research on Motion Simulation and Visual Recognition Algorithm of Guide Dog Walking Mechanism [J]. Electronic Science and Technology, 2021, 34(9): 66-72.
[14]	MA Lixin,DOU Chenfei,SONG Chencan,YANG Tianxiao. Insulator Nondestructive Testing Based on Feature Fusion CNN [J]. Electronic Science and Technology, 2021, 34(7): 26-30.
[15]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.