多尺度卷积结合Transformer的抑郁脑电分类研究

doi:10.19665/j.issn1001-2400.20230211

摘要/Abstract

摘要：

在通过深度学习模型进行抑郁症类脑电信号分析时,针对单一尺度的卷积存在特征提取不充分的问题和卷积神经网络在感知脑电信号全局依赖性方面的局限性,分别设计了多尺度动态卷积网络模块和门控Transformer编码器模块,并与时间卷积网络相结合,提出了混合网络模型(MGTTCNet)进行抑郁症患者和健康对照组的脑电信号分类。该模型首先通过多尺度动态卷积从空间域和频率域捕捉脑电信号的多尺度时频信息。其次通过门控Transformer编码器学习脑电信号中的全局依赖关系,其利用多头注意力机制有效增强网络表达相关脑电信号特征的能力。之后利用时间卷积网络提取脑电信号可用的时间特征,最后将提取的抽象特征输入到分类模块进行分类。在公开数据集MODMA上用留出法和十折交叉验证法对提出模型进行实验验证,分别取得了约98.51%和98.53%的分类准确率,相较于基线单尺度模型EEGNet,分类准确率分别提升了约1.89%和1.93%,F1值分别提升了约2.05%和2.08%,kappa系数值分别提高了约0.038 1和0.038 5;同时消融实验验证了文中设计的各个模块的有效性。

关键词: 脑电信号, 抑郁分类, 深度学习, Transformer, 时间卷积网络

Abstract:

In the process of using the deep learning model to classify the EEG signals of depression,aiming at the problem of insufficient feature extraction in single-scale convolution and the limitation of the convolutional neural network in perceiving the global dependence of EEG signals,a multi-scale dynamic convolution network module and the gated transformer encoder module are designed respectively,which are combined with the temporal convolution network,and a hybrid network model MGTTCNet is proposed to classify the EEG signals of patients with depression and healthy controls.First,multi-scale dynamic convolution is used to capture the multi-scale time-frequency information of EEG signals from spatial and frequency domains.Second,the gated transformer encoder is used to learn global dependencies in EEG signals,which effectively enhances the ability of the network to express relevant EEG signal features using the multi-head attention mechanism.Third,the temporal convolution network is used to extract temporal features available for EEG signals.Finally,the extracted abstract features are fed into the classification module for classification.The proposed model is experimentally validated on the public data set MODMA using the Hold-out method and the 10-Fold Cross Validation method,with the classification accuracy being 98.51% and 98.53%,respectively.Compared with the baseline single-scale model EEGNet,the classification accuracy of the proposed model is increased by 1.89% and 1.93%,the F1 value is increased by 2.05% and 2.08%,and the kappa coefficient values are increased by 0.0381 and 0.0385,respectively.Meanwhile,the ablation experiments verify the effectiveness of each module designed in this paper.

Key words: electroencephalography, depression classification, deep learning, Transformer, temporal convolutional networks

中图分类号:

TP391

翟凤文, 孙芳林, 金静. 多尺度卷积结合Transformer的抑郁脑电分类研究[J]. 西安电子科技大学学报, 2024, 51(2): 182-195.

ZHAI Fengwen, SUN Fanglin, JIN Jing. Study of EEG classification of depression by multi-scale convolution combined with the Transformer[J]. Journal of Xidian University, 2024, 51(2): 182-195.

图/表 18

图1

图2

图3

图4

表1

表2

表3

表4

表5

表6

图5

图6

表7

图7

表9

表10

图8

图9

参考文献 35

[1]	KARUNARATHNE A, GUNNELL D, KONRADSEN F, et al. How Many Premature Deaths From Pesticide Suicide Have Occurred Since the Agricultural Green Revolution?[J]. Clinical Toxicology, 2019, 58(4):227-232.
[2]	FRIEDRICH M J. Depressionis the Leading Cause of Disability Around the World[J]. JAMA, 2017, 317(15):1517.
[3]	SAEIDI M, KARWOWSKI W, FARAHANI F, et al. Neural Decoding of EEG Signals with Machine Learning:A Systematic Review[J]. Brain Sciences, 2021(11):1525.
[4]	SCHIRRMEISTER R, SPRINGENBERG J, FIEDERER L, et al. Deep Learning with Convolutional Neural Networks for EEG Decoding and Visualization[J]. Human Brain Mapping, 2017,38:5391-5420.
[5]	ACHARYA U, RAJENDRA S, YUKI H, et al. Automated EEG-Based Screening of Depression Using Deep Convolutional Neural Network[J]. Computer Methods and Programs in Biomedicine, 2018,161:103-113.
[6]	KARAKUS B, YILDIRIM Ö, TALO M, et al. Automated Depression Detection Using Deep Representation and Sequence Learning with EEG Signals[J]. Journal of Medical Systems, 2019,43:1-12.
[7]	UYULAN Ç, ERGÜZEL T, et al. UNUBOLH, Major Depressive Disorder Classification Based on Different Convolutional Neural Network Models:Deep Learning Approach[J]. Clinical EEG and Neuroscience, 2020,52:38-51.
[8]	DAN Y, ZHAO L L, SONG X W, et al. Automated Detection of Clinical Depression Based on Convolution Neural Network Model[J]. Biomedical Engineering/Biomedizinische Technik, 2022,67:131-142.
[9]	LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet:A Compact Convolutional Network for EEG-Based Brain-Computer Interfaces[J]. Journal of Neural Engineering, 2018, 15(5):1-17.
[10]	LIU W, JIA K B, WANG Z Z, et al. A Depression Prediction Algorithm Based on Spatiotemporal Feature of EEG Signal[J]. Brain Sciences, 2022, 12(5):630.
[11]	BAI S, KOLTER J, KOLTUN V. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling(2022)[J/OL].[2022-11-13]. https://arxiv.org/abs/1803.01271.
[12]	INGOLFSSON T M, HERSCHE M, WANG X, et al. EEG-TCNet:An Accurate Temporal Convolutional Network for Embedded Motor-Imagery Brain-Machine Interfaces[C]//Proceedings of the 2020 IEEE International Conference on Systems,Man,and Cybernetics(SMC). Piscataway:IEEE, 2020:2958-2965.
[13]	HASHEMPOUR S, BOOSTANI R, MOHAMMADI M, et al. Continuous Scoring of Depressionfrom EEG Signals via a Hybrid of Convolutional Neural Networks[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022,30:176-183.
[14]	王怡忻, 朱湘茹, 杨利军. 融合共空间模式与脑网络特征的EEG抑郁识别[J]. 计算机工程与应用, 2021, 58(22):150-158.
	WANG Yixin, ZHU Xiangru, YANG Lijun. EEG Depression Recognition Based on Feature Fusion of Common Spatial Pattern and Brain Connectivity[J]. Computer Engineering and Applications, 2021, 58(22):150-158.
[15]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is All You Need[C]// Proceedings of the in Advances in Neural Information Processing Systems(NIPS). San Diego: NIPS, 2017:5998-6008.
[16]	SONG Y H, JIA X Y, YANG L, et al. Transformer-Based Spatial-Temporal Feature Learning for EEG Decoding(2022)[J/OL].[2022-11-13]. https://arxiv.org/abs/2106.11170.
[17]	CASAL R, PERSIA L, SCHLOTTHAUER G. Temporal Convolutional Networks and Transformers for Classifying the Sleep Stage in Awake or Asleep Using Pulse Oximetry Signals[J]. Journal of Computational Science, 2022,59:101544.
[18]	MA Y, SONG Y, GAO F. A Novel Hybrid CNN-Transformer Model for EEG Motor Imagery Classification[C]// Proceedings of the IEEE International Joint Conference on Neural Network(IJCNN). Piscataway:IEEE, 2022:1-8.
[19]	张静, 张雪英, 陈桂军, 等. 结合3D-CNN和频-空注意力机制的EEG情感识别[J]. 西安电子科技大学学报, 2022, 49(3):191-198.
	ZHANG Jing, ZHANG Xueying, CHEN Guijun, et al. EEG Emotion Recognition Based on the 3D-CNN and Spatial-Frequency Attention Mechanism[J]. Journal of Xidian University, 2022, 49(3):191-198.
[20]	DAUPHIN Y, FAN A, AULI M, et al. Language Modeling with Gated Convolutional Networks[C]// Proceedings of the 34th International Conference on Machine Learning(ICML). New York: ACM, 2017:933-941.
[21]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway:IEEE, 2016:770-778.
[22]	CAI H S, YUAN Z Q, GAO Y P, et al. A Multi-Modal Open Dataset for Mental-Disorder Analysis[J]. Scientific Data, 2022,9:178-188.
[23]	ARBANAS G. Diagnostic and Statistical Manual of Mental Disorders(DSM-5)[J]. Alcoholism and psychiatry research, 2015,51:61-64.
[24]	SUN S T, LI J X, CHEN H Y, et al. A Study of Resting-State EEG Biomarkers for Depression Recognition(2022)[J/OL].[2022-10-23]. https://arxiv.org/abs/2002.11039.
[25]	SEAL A, BAJPAI R, AGNIHOTRI J, et al. DeprNet:A Deep Convolution Neural Network Framework for Detecting Depression Using EEG[J]. IEEE Transactions on Instrumentation and Measurement, 2021,70:1-13.
[26]	JAS M, ENGEMANN D, BEKHTI Y, et al. Autoreject:Automated Artifact Rejection for MEG and EEG Data[J]. Neuroimage, 2016,159:417-429.
[27]	HASANZADEH F, MOHEBBI M, ROSTAMI R. Graph Theory Analysis of Directed Functional Brain Networks in Major Depressive Disorder Based on EEG Signal[J]. Journal of Neural Engineering, 2020, 17(2):026010.
[28]	SUN S T, CHEN H Y, SHAO X X, et al. EEG Based Depression Recognition by Combining Functional Brain Network and Traditional Biomarkers[C]//Proceedings of the 2020 IEEE International Conference on Bioinformatics and Biomedicine(BIBM). Piscataway:IEEE, 2020:2074-2081.
[29]	尚照岩, 乔晓艳. 轻度抑郁症脑电特征分析与机器识别研究[J]. 测试技术学报, 2022, 36(6):498-505.
	SHANG Zhaoyan, QIAO Xiaoyan. Study on Analysis and Recognition of EEG Characteristics of Mild Depression[J]. Journal of Test and Measurement Technology, 2022, 36(6):498-505.
[30]	SANTAMARIA V E, MARTINEZ C V, VAQUERIZO V F, et al. EEG-Inception:A Novel Deep Convolutional Neural Network for Assistive ERP-Based Brain-Computer Interfaces[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(12):2773-2782.
[31]	SALAMI A, ANDREU-PEREZ J, GILLMEISTER H. EEG-ITNet:An Explainable Inception Temporal Convolutional Network for Motor Imagery Classification[J]. IEEE Access, 2022,10:36672-36685.
[32]	ZHANG X W, LI J L, HOU K C, et al. EEG-Based Depression Detection Using Convolutional Neural Network with Demographic Attention Mechanism[C]//Proceedings of the annual International Conference of the IEEE Engineering in Medicine and Biology Society(EMBC). Piscataway:IEEE, 2020:128-133.
[33]	WANG Y X, LIU F R, YANG L J. EEG-Based Depression Recognition Using Intrinsic Time-scale Decomposition and Temporal Convolution Network[C]//Proceedings of the International Conference on Biological Information and Biomedical Engineering(BIBE). New York: ACM, 2021:1-6.
[34]	DENG X, FAN X F, LV X W, et al. SparNet:A Convolutional Neural Network for EEG Space-Frequency Feature Learning and Depression Discrimination[J]. Frontiers in Neuroinformatics, 2022,16:914823.
[35]	MAATEN L, HINTON G. Visualizing Data Using t-SNE[J]. Journal of Machine Learning Research, 2008,9:2579-2605.

验证方法	模型	Acc/%	F1/%	Sens/%	Spec/%	kappa
	SiNet	90.87	90.24	89.91	91.72	0.816 7
	MsNet	95.33	95.08	96.11	94.65	0.906 5
Hold-out	MGTNet	96.55	96.33	96.40	96.69	0.930 8
	MTCNet	96.89	96.74	98.27	95.67	0.937 7
	MGTTCNet	98.51	98.42	98.85	98.22	0.970 2
	MTCNGTNet	97.63	97.45	96.54	98.60	0.952 4

验证方法	模型	Acc/%	F1/%	Sens/%	Spec/%	kappa
	SiNet	92.62±1.37	92.21±1.49	92.51±2.36	92.72±1.63	0.852 0±0.027 6
	MsNet	94.76±0.70	94.43±0.78	94.09±2.00	95.36±1.56	0.894 8±0.014 1
10-Fold CV	MGTNet	96.10±0.59	95.91±0.62	96.69±1.21	95.58±1.19	0.921 9±0.011 8
	MTCNet	97.44±0.37	97.29±0.39	97.19±0.58	97.67±0.62	0.948 7±0.007 5
	MGTTCNet	98.53±0.40	98.45±0.42	98.73±0.51	98.36±0.66	0.970 6±0.008 1
	MTCNGTNet	98.24±0.48	98.14±0.45	98.28±0.49	98.20±0.58	0.964 7±0.007 2

验证方法	频带	Acc/%	F1/%	Sens/%	Spec/%	kappa
	delta	90.59	89.52	87.81	92.94	0.809 9
	theta	94.65	94.14	94.21	95.02	0.892 1
Hold-out	alpha	96.51	97.18	96.91	97.69	0.946 2
	beta	97.87	97.78	97.99	97.82	0.957 9
	gamma	97.01	97.25	96.56	97.19	0.948 1

验证方法	频带	Acc/%	F1/%	Sens/%	Spec/%	kappa
	delta	89.95±0.48	89.19±0.52	87.69±2.54	91.97±2.53	0.798 0±0.009 4
	theta	94.10±0.80	93.70±0.87	93.24±1.67	94.82±1.29	0.881 6±0.016 1
10-Fold CV	alpha	96.79±0.41	96.81±0.44	96.44±0.76	97.49±0.66	0.939 6±0.008 2
	beta	97.62±0.35	97.53±0.35	97.71±0.82	97.54±0.95	0.952 4±0.006 9
	gamma	97.21±0.34	97.07±0.36	96.18±0.75	97.81±0.73	0.944 1±0.006 9

验证方法	模型名称	Acc/%	F1/%	Sens/%	Spec/%	kappa
	EEGNet-8,2^[9]	96.62	96.37	95.68	97.45	0.932 1
	EEG-TCNet^[12]	94.86	94.48	93.80	95.80	0.896 8
Hold-out	EEG-Inception^[30]	95.27	94.88	93.37	96.94	0.904 8
	EEG-ITNet^[31]	91.41	91.24	95.24	88.03	0.828 5
	MGTTCNet	98.51	98.42	98.85	98.22	0.970 2