特征融合实现脑电信号情感分析

doi:10.19665/j.issn1001-2400.2022.06.012

摘要/Abstract

摘要：

由于脑电信号具有非平稳、微弱、频段差异大的特点,在处理过程中难以获得较好的识别精度。为了提高脑电情感分析的性能,从特征提取和特征选择两个方面进行了探讨。在特征提取方面,在功率谱强度的基础上采用改进的平衡功率谱强度特征;在特征选择方面,提出一种特征融合算法,充分利用Relief和mRMR两种方法各自的优势,在提高识别性能的同时大幅度地降低了特征维度。采用支持向量机分类算法,在DEAP数据集上进行了实验。结果表明,相比于功率谱强度,平衡功率谱强度的分类准确率平均提高了6.22%,特征融合算法选择的特征相比于基线提高了3.90分,相比于Relief提高了1.84分,相比于 mRMR提高了2.05分。基于平衡功率谱强度的特征融合算法,不仅其整体性能提升了,且其平均识别准确率在Valence维度上达到88.89%,在Arousal维度上达到87.73%,同时平均特征维度从160维减少至67维。

关键词: 情感识别, 情感计算, 脑电信号, 特征提取, 融合特征, 特征选择

Abstract:

Since the EEG signal has the characteristics of non-stationary,weak,and large frequency difference,it is difficult to obtain a higher recognition accuracy.In order to improve the performance of EEG sentiment analysis,this paper conducts research from two aspects:feature extraction and feature selection.In terms of feature extraction,based on the power spectrum intensity,the balanced power spectrum intensity feature (BPSI) is adopted.For feature selection,a feature fusion algorithm FFS is proposed,which combines the Relief and mRMR to reduce the feature dimension and improve the recognition performance.This paper uses the SVM classification algorithm,and carries out experiments on DEAP data.Experimental results show that,compared with the power spectrum intensity,the classification accuracy of the BPSI feature is increased by 6.22% on average.The performance is increased by 3.9 points with features selected by the FFS compared with the baseline,by 1.84 points compared with the Relief,and by 2.05 points compared with the mRMR.The average accuracy of the emotion recognition algorithm based on the BPSI and FFS reaches 88.89% in Valence dimension and 87.73% in Arousal dimension,and meanwhile the average feature dimension is reduced from 160 to 67.

Key words: emotion recognition, affective computing, electro encephalo gram signal, feature extraction, feature fusion, feature selection

中图分类号:

TP181

杨利英,孟天昊,张清杨,晁思. 特征融合实现脑电信号情感分析[J]. 西安电子科技大学学报, 2022, 49(6): 95-102.

YANG Liying,MENG Tianhao,ZHANG Qingyang,CHAO Si. Implementation of EEG emotion analysis via feature fusion[J]. Journal of Xidian University, 2022, 49(6): 95-102.

图/表 4

表1

表2

图1

表3

参考文献 24

[1]	CHAPMAN B P, FISCELLA K, KAWACHI I, et al. Emotion Suppression and Mortality Risk over a 12-Year Follow-up[J]. Journal of Psychosomatic Research, 2013, 75(4):381-385. doi: 10.1016/j.jpsychores.2013.07.014 pmid: 24119947
[2]	PICARD R W. Affective Computing[M]. Cambridge: MIT Press, 2000:129-142.
[3]	MOORE R K. Towards a Unified Theory of Spoken Language Processing[C]// Proceedings of the 4th IEEE International Conference on Cognitive Informatics (ICCI 2005).Piscataway:IEEE, 2005:167-172.
[4]	KHALILI Z, MORADI M H. Emotion Recognition System Using Brain and Peripheral Signals:Using Correlation Dimension to Improve the Results of EEG[C]// International Joint Conference on Neural Networks.Piscataway:IEEE, 2009:1571-1575.
[5]	郭柳君, 张雪英, 陈桂军. 深度线性判别分析用于两级脑控字符拼写解码[J]. 西安电子科技大学学报, 2020, 47(4):109-116.
	GUO Liujun, ZHANG Xueying, CHEN Guijun. Deep Linear Discriminant Analysis for Two-level Brain-controlled Character Spelling Decoding[J]. Journal of Xidian University, 2020, 47(4):109-116.
[6]	柳素红, 孙晓, 李春彬. 基于位置信息重建与时频域信息融合的脑电信号情感识别[J]. 计算机工程, 2021, 47(12):95-102.
	LIU Suhong, SUN Xiao, LI Chunbin. Emotion Recognition of EEG Signals based on Location Information Reconstruction and Time-frequency Information Fusion[J]. Computer Engineering, 2021, 47(12):95-102.
[7]	ZHENG X, LIU X, ZHANG Y, et al. A Portable HCI System-Oriented EEG Feature Extraction and Channel Selection for Emotion Recognition[J]. International Journal of Intelligent Systems, 2021, 36(1):152-176. doi: 10.1002/int.22295
[8]	马江河, 孙颖, 张雪英. 融合语音信号和脑电信号的多模态情感识别[J]. 西安电子科技大学学报, 2019, 46(1):143-150.
	MA Jianghe, SUN Ying, ZHANG Xueying. Multimodal Emotion Recognition for the Fusion of Speech and EEG Signals[J]. Journal of Xidian University, 2019, 46(1):143-150.
[9]	ZHANG Wei,YIN Zhong. EEG Feature Selection for Emotion Recognition Based on Cross-subject Recursive Feature Elimination[C/OL].[2021-11-30].DOI:10.26914/c.cnkihy.2020.038294. doi: 10.26914/c.cnkihy.2020.038294
[10]	XU X, WEI F, ZHU Z, et al. EEG Feature Selection Using Orthogonal Regression:Application to Emotion Recognition[C]// 2020 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).Piscataway:IEEE, 2020:1239-1243.
[11]	ASGHAR M A, KHAN M J, SHAHID H, et al. Semi-Skipping Layered Gated Unit and Efficient Network:Hybrid Deep Feature Selection Method for Edge Computing in EEG-Based Emotion Classification[J]. IEEE Access, 2021,(99):1.
[12]	曹卫东, 李嘉琪, 王怀超. 采用注意力门控卷积网络模型的目标情感分析[J]. 西安电子科技大学学报, 2019, 46(6):30-36.
	CAO Weidong, LI Jiaqi, WANG Huaichao. Analysis of Targeted Sentiment by the Attention Gated Convolutional Network Model[J]. Journal of Xidian University, 2019, 46(6):30-36.
[13]	NAWAZ R, CHEAH K H, NISAR H, et al. Comparison of Different Feature Extraction Methods for EEG-Based Emotion Recognition[J]. Biocybernetics and Biomedical Engineering, 2020, 40(3):910-926. doi: 10.1016/j.bbe.2020.04.005
[14]	LIU J, MENG H, NANDI A, et al. Emotion detection from EEG recordings[C]// 2016 12th International Conference on Natural Computation,Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).Piscataway:IEEE, 2016:1722-1727.
[15]	CANDRA H, YUWONO M, CHAI R, et al. EEG Emotion Recognition Using Reduced Channel Wavelet Entropy and Average Wavelet Coefficient Features with Normal Mutual Information Method[C]// International Conference of the IEEE Engineering in Medicine & Biology Society.Piscataway:IEEE, 2017:463-466.
[16]	SOROUSH M Z, MAGHOOLI K, SETAREHDAN S K, et al. A Novel EEG-Based Approach to Classify Emotions Through Phase Space Dynamics[J]. Signal,Image and Video Processing, 2019, 13(6):1149-1156. doi: 10.1007/s11760-019-01455-y
[17]	ISMAEL A M, ALIN M F, ABDALLA K H, et al. Two-Stepped Majority Voting for Efficient EEG-Based Emotion Classification[J]. Brain Informatics, 2020, 7(1):1-12. doi: 10.1186/s40708-020-0102-9
[18]	KOELSTRA S, MUHL C, SOLEYMANI M, et al. DEAP:A Database for Emotion Analysis;Using Physiological Signals[J]. IEEE Transactions on Affective Computing, 2012, 3(1):18-31. doi: 10.1109/T-AFFC.2011.15
[19]	杨静波, 赵启军, 吕泽均. 维度情感模型下的表情图像生成及应用[J]. 西安电子科技大学学报, 2021, 48(5):30-37.
	YANG Jingbo, ZHAO Qijun, LV Zejun. Synthesis of the Expression Image and Its Application under the Dimentional Emotion Model[J]. Journal of Xidian University, 2021, 48(5):30-37.
[20]	SHI L C, JIAO Y Y, LU B L. Differential Entropy Feature for EEG-Based Vigilance Estimation[C]// 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).Piscataway:IEEE, 2013:6627-6630.
[21]	BAO F S, LIU X, ZHANG C. PyEEG:an Open Source Python Module for EEG/MEG Feature Extraction[J]. Computational Intelligence and Neuroscience, 2011, 2011:406391-406397.
[22]	KIRA K, RENDELL L A. The Feature Selection Problem:Traditional Methods and a New Algorithm[C]// Proceedings of the Tenth National Conference on Artificial Intelligence. New York: AAAI, 1992:129-134.
[23]	PENG H, LONG F, DING C. Feature Selection Based on Mutual Information Criteria of Max-Dependency,Max-Relevance,and Min-Redundancy[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2005, 27(8):1226-1238.
[24]	YAN J, CHEN S, DENG S. A EEG-Based Emotion Recognition Model with Rhythm and Time Characteristics[J]. Brain Informatics, 2019, 6(1):1-8. doi: 10.1186/s40708-019-0095-4

项目	描述
受试者	32(16位男士,16位女士)
通道数	40(32个脑电信号,8个其他生理信号)
影片长度	3 s(准备时间),60 s(刺激时间)
标签类型	Valence、Arousal、Dominance、Liking
采样频率/Hz	128

特征	Valence维度	Arousal维度
ESD	74.52	70.75
PSD	74.88	70.64
RIR	85.64	85.06
DE	87.72	87.38
PSI	82.78	80.99
BPSI	88.14	88.07

方法	Valence维度	Arousal维度
mRMR^[14] (2016)	69.90	71.20
NMI^[15](2017)	76.80	74.30
庞加莱交叉点^[16](2019)	87.42	84.59
两部投票 ^[17] (2020)	85.00	86.30
文中算法	88.89	87.73