结合共注意网络的深度BiGRU和DPCS情感分析模型

doi:10.16180/j.cnki.issn1007-7820.2025.05.004

摘要/Abstract

摘要：

针对一词多义现象和情感分析模型无法提取全面深度语义特征等问题,文中提出结合共注意网络的深度BiGRU(Bidirectional Gated Recurrent Unit)和DPCS(Deep Convolutional Attention Networks)情感分析模型。使用RoBERTaRoBERTa(Robustly optimized BERT approach)获取文本的动态语义表征,通过并行双通道网络的深度BiGRU与DPCS分别提取深层次文本上下文语义特征和重要文本局部特征,利用基于共注意网络的特征融合将不同方面文本语义特征进行深度融合以获取更全面深层次的全局语义特征。为验证所提模型的有效性,在电影和线上购物评论数据集上进行实验对比。实验结果表明,所提模型的准确率和F1均高于其他模型,在两个数据集上准确率分别达到了93.05%和94.67%。

关键词: 文本情感分析, RoBERTa, 双向门控循环神经网络, 自注意力机制, 卷积神经网络, 动态共注意力网络, 特征融合, 全局语义特征

Abstract:

In view of the problem of polysemous phenomena and the inability of emotion analysis model to extract comprehensive deep semantic features, this paper proposes a deep BiGRU(Bidirectional Gated Recurrent Unit) and DPCS(Deep Convolutional Attention Networks) emotion analysis model combined with coattention network. The model uses RoBERTa(Robustly optimized BERT approach) to obtain dynamic semantic representation of text, extracts deep contextual semantic features and important local text features through parallel dual-channel network deep BiGRU and DPCS, and uses co-attention network-based feature fusion to deeply integrate different aspects of text semantic features to obtain more comprehensive and deep global semantic features. In order to verify the validity of the proposed model, an experimental comparison is performed on the data set of movie and online shopping reviews. The experimental results show that the accuracy and F1 of the proposed model are higher than other models, and the accuracy of the two data sets reaches 93.05% and 94.67%, respectively.

Key words: text sentiment analysis, RoBERTa, bidirectional gated recurrent unit, self-attention mechanism, convolutional neural network, dynamic coattention network, feature fusion, global semantic feature

中图分类号:

TP391

陈漫漫, 于莲芝. 结合共注意网络的深度BiGRU和DPCS情感分析模型[J]. 电子科技, 2025, 38(5): 22-30.

CHEN Manman, YU Lianzhi. Deep BiGRU and DPCS Sentiment Analysis Model Combined with Coattention Network[J]. Electronic Science and Technology, 2025, 38(5): 22-30.

图/表 10

图1

图2

图3

图4

表1

表2

表3

表4

表5

表6

参考文献 24

[1]	邓磊, 孙培洋. 基于深度学习的网络舆情监测系统研究[J]. 电子科技, 2022, 35(12): 97-102.
	Deng Lei, Sun Peiyang. Research on network public opinion monitoring system based on deep learning[J]. Electronic Science and Technology, 2022, 35(12):97-102.
[2]	王颖洁, 朱久祺, 汪祖民, 等. 自然语言处理在文本情感分析领域应用综述[J]. 计算机应用, 2022, 42(4):1011-1020. doi: 10.11772/j.issn.1001-9081.2021071262
	Wang Yingjie, Zhu Jiuqi, Wang Zumin, et al. Review of applications of natural language processing in text sentiment analysis[J]. Journal of Computer Applications, 2022, 42(4):1011-1020. doi: 10.11772/j.issn.1001-9081.2021071262
[3]	Xu G, Yu Z, Yao H, et al. Chinese text sentiment analysis based on extended sentiment dictionary[J]. IEEE Access, 2019, 7(3):43749-43762.
[4]	Kaur S, Sikka G, Awasthi L K. Sentiment analysis approach based on N-gram and KNN classifier[C]. Jalandhar: First International Conference on Secure Cyber Computing and Communication, 2018:1-4.
[5]	Kim H, Jeong Y S. Sentiment classification using convolutional neural networks[J]. Applied Sciences, 2019, 9(11):1-14.
[6]	Wang X, Jiang W, Luo Z. Combination of convolutional and recurrent neural network for sentiment analysis of short texts[C]. Osaka: Proceedings of the Twenty-sixth International Conference on Computational Linguistics, 2016:2428-2437.
[7]	Xiong C, Zhong V, Socher R. Dynamic coattention networks for question answering[EB/OL].(2018-3-6)[2023-11-24].https://doi.org/10.48550/arXiv.1611.01604.
[8]	Alzahrani M E, Aldhyani T H H, Alsubari S N, et al. Developing an intelligent system with deep learning algorithms for sentiment analysis of ecommerce product reviews[J]. Computational Intelligence and Neuroscience, 2022, 22(5):1-10.
[9]	Mahto D, Yadav S C, Lalotra G S. Sentiment prediction of textual data using hybrid ConvBidirectional-LSTM model[J]. Mobile Information Systems, 2022, 37(6):1-11.
[10]	Li X Y, Raga R C, Shi X M. GloVe-CNN-BiLSTM model for sentiment analysis on text reviews[J]. Journal of Sensors, 2022, 24(10):1-12.
[11]	杨森淇, 段旭良, 肖展, 等. 基于ERNIE+DPCNN+BiGRU的农业新闻文本分类[J]. 计算机应用, 2023, 43(5):1461-1466. doi: 10.11772/j.issn.1001-9081.2022040641
	Yang Senqi, Duan Xuliang, Xiao Zhan, et al. Text classification of agricultural news based on ERNIE+DPCNN+BiGRU[J]. Journal of Computer Applications, 2023, 43(5):1461-1466. doi: 10.11772/j.issn.1001-9081.2022040641
[12]	杨春霞, 徐奔, 陈启岗, 等. 融合深度BiGRU与全局图卷积的方面级情感分析模型[J]. 小型微型计算机系统, 2023, 44(1):132-139.
	Yang Chunxia, Xu Ben, Chen Qigang, et al. Aspect level sentiment analysis model based on deep BiGRU and global graph convolution[J]. Journal of Chinese Computer Systems, 2023, 44(1):132-139.
[13]	陈虎, 高波涌, 陈莲娜, 等. 结合注意力机制与双向切片GRU的情感分类模型[J]. 小型微型计算机系统, 2020, 41(9):1793-1799.
	Chen Hu, Gao Boyong, Chen Lianna, et al. Sentiment classification model combining attention mechanism and bidirectional slice GRU[J]. Journal of Chinese Computer Systems, 2020, 41(9):1793-1799.
[14]	Peng H, Zhang Z, Liu H. A sentiment analysis method for teaching evaluation texts using attention mechanism combined with CNN-BLSTM model[J]. Scientific Programming, 2022, 31(2):1-9.
[15]	Yan W, Zhou L, Qian Z, et al. Sentiment analysis of student texts using the CNN-BiGRU-AT model[J]. Scientific Programming, 2021, 30(10):17-19.
[16]	李辉, 王一丞. 基于CNNCIFG-Attention模型的文本情感分类[J]. 电子科技, 2022, 35(2):46-51.
	Li Hui, Wang Yicheng. CNNCIFG-Attention model for text sentiment classification[J]. Electronic Science and Technology, 2022, 35(2):46-51.
[17]	Long F, Zhou K, Ou W. Sentiment analysis of text based on bidirectional LSTM with multi-head attention[J]. IEEE Access, 2019, 7(9):141960-141969.
[18]	Devlin J, Chang M W, Lee K, et al. BERT:Pretraining of deep bidirectional transformers for language understanding[EB/OL].(2019-5-24)[2023-11-24].https://doi.org/10.48550/arXiv.1810.04805.
[19]	Liu Y, Ott M, Goyal N, et al. RoBERTa:A robustly optimized bert pretraining approach[EB/OL]. (2019-7-26)[2023-11-24].https://doi.org/10.48550/arXiv.1907.11692.
[20]	Xu M, Wang D, Feng S, et al. KC-ISA:An implicit sentiment analysis model combining knowledge enhancement and context features[C]. Gyeongju: Proceedings of the Twenty-ninth International Conference on Computational Linguistics, 2022: 6906-6915.
[21]	Li B, Pan F, Shou Z, et al. Aspect based sentiment analysis of catering field reviews via RoBERTa-AOA model[C]. Sanya: The Fourth International Conference on Advanced Algorithms and Control Engineering, 2021: 1-7.
[22]	Johnson R, Zhang T. Deep pyramid convolutional neural networks for text categorization[C]. Vancouver: Proceedings of the Fifty-fifth Annual Meeting of the Association for Computational Linguistics, 2017: 562-570.
[23]	Liu G, Guo J. Bidirectional LSTM with attention mechanism and convolutional layer for text classification[J]. Neurocomputing, 2019, 337(4):325-338.
[24]	Basiri M E, Nemati S, Abdar M, et al. ABCDM:An attention-based bidirectional CNN-RNN deep model for sentiment analysis[J]. Future Generation Computer Systems, 2021, 115(9):279-294.

名称	电影评论数据集	线上购物评论数据集
Train_data	34 255	50 068
Dev_data	3 823	3 769
Test_data	2 002	1 953

参数项目	名称
操作系统	Ubuntu
显卡	Intel(R) Xeon(R)
编程语言	Python 3.8.10
深度学习框架	Pytorch 1.12.0
CUDA版本	11.3

超参数名称	设置数值
sentence length	70
word_vector	768
BiGRU hidden	64
filter sizes	3
num filters	128
Self-Attention out_dim	128
dropout	0.4
max epochs	20
优化器	AdamW
learning rate	0.000 07
batch size	128

模型	电影评论数据集		线上购物评论数据集
模型	Accuracy/%	F1/%	Accuracy/%	F1/%
BiGRU	89.35	89.00	92.42	92.44
CRNN	84.05	83.29	91.09	91.08
DPCNN	80.10	79.53	87.56	87.63
AC-BiLSTM	84.65	84.22	91.04	90.97
ABCDM	90.85	90.48	93.14	93.18
DBGDCN_DPCS	93.05	93.06	94.67	94.81

模型	电影评论数据集		线上购物评论数据集
模型	Accuracy/%	F1/%	Accuracy/%	F1/%
深度BiGRU单通道	92.65	92.70	94.01	94.08
DPCS单通道	92.90	92.85	94.06	94.25
DBG_DPCS	92.25	92.46	94.52	94.60
DBGDCN_DPCS	93.05	93.06	94.67	94.81