基于CNNCIFG-Attention模型的文本情感分类

doi:10.16180/j.cnki.issn1007-7820.2022.02.008

摘要/Abstract

摘要：

神经网络在处理中文文本情感分类任务时,文本显著特征提取能力较弱,学习速率也相对缓慢。针对这一问题,文中提出一种基于注意力机制的混合网络模型。首先对文本语料进行预处理,利用传统的卷积神经网络对样本向量的局部信息进行特征提取,并将其输入耦合输入和遗忘门网络模型,用以学习前后词句之间的联系。随后,再加入注意力机制层,对深层次文本信息进行权重分配,提高重要信息对文本情感分类的影响强度。最后,将所提出的混合网络模型在京东商品评论集上进行测试。测试结果显示,新方法的准确率达到了92.13%,F-Score数值为92.06%,证明了 CNNCIFG-Attention 模型的可行性。

关键词: 情感分类, 混合网络模型, 卷积神经网络, 特征提取, 耦合输入和遗忘门网络, 注意力机制, 权重分配, 准确率, F-Score数值

Abstract:

Neural networks are weak in text salient feature extraction and have relatively slow learning rate in processing Chinese text sentiment classification tasks. To solve this problem, this study proposes a hybrid network model based on attention mechanism. This study preprocesses the text corpus, uses the traditional convolutional neural network to extract the feature of the local information of the sample vector. Then, extracted features are input into the coupled input and forget gate network model to learn the connection between the preceding and following words and sentences. Subsequently, the attention mechanism layer is added to assign weights to deep-level text information to improve the intensity of the influence of important information on text sentiment classification. Finally, the proposed hybrid network model is tested on the crawled JD product review collection. The test results show that the accuracy of the proposed method reaches 92.13%, and the F-Score value is 92.06%, which proves the feasibility of the CNNCIFG-Attention model.

Key words: sentiment classification, hybrid network model, convolutional neural network, feature extraction, coupled input and forget gate network, attention model, weight distribution, accuracy, F-Score value

中图分类号:

TP391.1

李辉,王一丞. 基于CNNCIFG-Attention模型的文本情感分类[J]. 电子科技, 2022, 35(2): 46-51.

LI Hui,WANG Yicheng. CNNCIFG-Attention Model for Text Sentiment Classifcation[J]. Electronic Science and Technology, 2022, 35(2): 46-51.

图/表 8

图1

图2

图3

表1

表2

表3

图4

图5

参考文献 18

[1]	Dragoni M. A three-phase approach for exploiting opinion mining in computational advertising[J]. IEEE Intelligent Systems, 2017, 32(3):21-27. doi: 10.1109/MIS.2017.46
[2]	房乐楠, 何腾鹏, 刘宇红. 一种改进型PSO算法在SVM参数寻优中的应用[J]. 电子科技, 2018, 31(6):17-19.
	Fang Lenan, He Tengpeng, Liu Yuhong. Application of an improved PSO algorithm in SVM parameter optimization[J]. Electronic Science and Technology, 2018, 31(6):17-19.
[3]	Bilal M, Israr H, Shahid M, et al. Sentiment classification of Roman-Urdu opinions using Nave Bayesian, decision tree and KNN classification techniques[J]. Journal of King Saud University-Computer and Information Sciences, 2016, 28(3):330-344. doi: 10.1016/j.jksuci.2015.11.003
[4]	Huang M L, Qian Q, Zhu X Y. Encoding syntactic knowledge in neural networks for sentiment classification[J]. ACM Transactions on Information Systems, 2017, 35(3):1-27.
[5]	Poria S, Peng H Y, Hussain A, et al. Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis[J]. Neurocomputing, 2017, 26(1):217-230.
[6]	Stojanovski D, Strezoski G, Madjarov G, et al. Deep neural network architecture for sentiment analysis and emotion identification of Twitter messages[J]. Multimedia Tools and Applications, 2018, 77(24):32213-32242. doi: 10.1007/s11042-018-6168-1
[7]	梁斌, 刘全, 徐进, 等. 基于多注意力卷积神经网络的特定目标情感分析[J]. 计算机研究与发展, 2017, 54(8):1724-1735.
	Liang Bin, Liu Quan, Xu Jin, et al. Aspect-based sentiment analysis based on multi-attention CNN[J]. Journal of Computer Research and Development, 2017, 54(8):1724-1735.
[8]	Rehman A U, Malik A K, Raza B, et al. A hybrid CNN-LSTM model for improving accuracy of movie reviews sentiment analysis[J]. Multimedia Tools and Applications, 2019, 78(18):26597-26613. doi: 10.1007/s11042-019-07788-7
[9]	Shuang K, Ren X T, Yang Q Q, et al. AELA-DLSTMs:attention-enabled and location-aware double LSTMs for aspect-level sentiment classification[J]. Neurocomputing, 2019, 33(4):25-34.
[10]	王伟, 王洪伟, 孟园. 协同过滤推荐算法研究:考虑在线评论情感倾向[J]. 系统工程理论与实践, 2014, 34(12):3238-3249.
	Wang Wei, Wang Hongwei, Meng Yuan. The collaborative filtering recommendation based on sentiment analysis of online reviews[J]. Systems Engineering-Theory & Practice, 2014, 34(12):3238-3249.
[11]	张平平. 卷积CIFG模型在影评情感分析中的应用[D]. 上海:上海师范大学, 2019.
	Zhang Pingping. Application of convolutional CIFG model in sentiment analysis of film review[D]. Shanghai:Shanghai Normal University, 2019.
[12]	Liu M F, Hu H J, Li L J, et al. Chinese image caption generation via visual attention and topic modeling[J]. IEEE Transactions on Cybernetics, 2020, 9(9):1-11.
[13]	Choi H, Cho K, Bengio Y. Fine-grained attention mechanism for neural machine translation[J]. Neurocomputing, 2018, 284(5):171-176. doi: 10.1016/j.neucom.2018.01.007
[14]	Zhou C T, Sun C L, Liu Z Y, et al. A C-LSTM neural network for text classification[J]. Computer Science, 2015, 1(4):39-44.
[15]	Li B Y, Zhou K M, Gao W, et al. Attention-based LSTM-CNNs for uncertainty identification on Chinese social media texts[C]. Shenzhen:Proceedings of the International Conference on Security,Pattern Analysis,and Cybernetics, 2017.
[16]	Zhang Y S, Jiang Y R, Tong Y X. Study of sentiment classification for Chinese microblog based on recurrent neural network[J]. Chinese Journal of Electronics, 2016, 25(4):601-607. doi: 10.1049/cje.v25.4
[17]	Dawn E H, Lakhmi C J. Innovations in machine learning[M]. Heidelberg:Springer, 2006.
[18]	Church W K. Word2vec[J]. Natural Language Engineering, 2017, 23(1):155-162. doi: 10.1017/S1351324916000334

数据集	样本情感属性	样本数量	句子平均长度
京东商品评论	积极	10 435	70
京东商品评论	消极	10 221	70
豆瓣电影评论	积极	12 942	240
豆瓣电影评论	消极	9 878	240

模型参数	取值
词向量维度	128
滤波器尺寸	3,4,5
滤波器个数	100,100,100
CIFG单元数量	128
Dropout率	0.5
Batch_size	64
迭代次数	20

数据集	模型	准确率	F-Score
京东商品评论	Naive Byes	85.26CNNCIFG	85.16CNNCIFG
	SVM	86.08CNNCIFG	86.18CNNCIFG
	CNN	87.31CNNCIFG	87.28CNNCIFG
	CNN-LSTM	89.14CNNCIFG	89.03CNNCIFG
	CNN-CIFG	90.02CNNCIFG	89.88CNNCIFG
	CNNCIFG-Attention	92.13CNNCIFG	92.06CNNCIFG
豆瓣电影评论	Naive Byes	85.11CNNCIFG	85.21CNNCIFG
	SVM	86.04CNNCIFG	86.14CNNCIFG
	CNN	87.20CNNCIFG	87.19CNNCIFG
	CNN-LSTM	88.00CNNCIFG	87.88CNNCIFG
	CNN-CIFG	88.52CNNCIFG	88.46CNNCIFG
	CNNCIFG-Attention	89.75CNNCIFG	89.68CNNCIFG