基于半监督深度生成对抗学习的脑电信号情感识别模型与方法研究

Most traditional researches on emotion recognition based on EEG applied the manual feature extraction and simple machine learning algorithms, which are difficult to extract the temporal and special invariant features of EEG signals at the same time and always consume a lot of time and machine memory not applicable in reality. Deep learning algorithms for EEG emotion recognition is also faced with the lack of labeled data and instability resulted by changes of EEG with time, individual and external environment. Therefore, the project studies and applies the deep learning models based on Long short time memory (LSTM) network, the convolutional neural network (CNN), and CNN combined with LSTM respectively, to end-to-end self-learn the discriminative features of EEG data invariant in time and space. A semi-supervised generative adversarial network is explored to combine EEG feature learning, sample generation and discriminant classification into a network system and train the whole network with an adversarial learning method, and finally improve the quality of generated data and the performance of EEG emotion classification. The project additionally proposes a domain-adversarial training deep learning neural network which jointly train the feature extractor, domain classifier and label classifier on labeled data from the source domain and unlabeled data from the target domain with a domain-adversarial training method to solve the domain adaption problem. The research of the project is of great theoretical and practical value for realization of advanced human-computer interaction and brain-computer interface system.

传统的脑电情感识别方法大都采用人工特征提取与简单分类算法相结合，很难同时提取EEG信号时间与空间上的相关特征，需要消耗大量的时间和内存。利用深度学习进行EEG情感识别面临着带标签数据不足，脑电信号随时间、个体及外部环境变化而变化导致的分类准确率和稳定性低等问题。对此，本文研究并利用LSTM、CNN以及CNN与LSTM相结合的深度学习神经网络，端对端学习EEG数据时间与空间上的相关特征。利用基于半监督模式的深度生成对抗网络，将EEG特征学习、样本生成和判别分类融合到一个网络体系中，以对抗训练方法同步学习整个网络，提高生成数据的质量和EEG情感分类的性能。探索一种基于域对抗训练的深度神经网络，通过域对抗方法同步训练EEG特征提取器、域分类器和标签分类器，解决EEG情感分类中训练数据和测试数据相似但分布不同的域自适应问题。项目的研究对于实现高级人机交互与脑-机接口系统具有重要理论意义。

神经学、心理学诸多研究成果表明，人类情感与大脑皮层活动密切相关。利用脑电波（EEG）能够客观准确测量人类的情感动态。针对传统脑电情感识别方法中手工特征提取效率低的问题，本项目首先提出了一种端对端的深度CNN网络，从脑电信号时域与频域组合特征中学习具有高判别性的深层特征；接着提出了一种分层注意力LSTM神经网络，在时间点和时间片段分别采用注意力机制，从非平稳EEG序列中学习深层情感相关性特征；针对多通道脑电信号在空间上的非平稳性和相关性，采用一种新的二维网状脑电数据表示方法，构建了级联与并联混合的卷积循环神经网络，联合学习二维网状EEG序列中的深层时空相关性特征；在此基础上提出了一种从局部到全局的领域对抗神经网络，进行分层时空脑电特征学习与情感识别，进一步提高了模型的精度和领域自适应能力；最后，针对EEG小样本问题提出了一种基于半监督模式的深度生成对抗网络，将EEG特征学习、样本生成和判别融合到一个网络体系中，以对抗训练方法同步学习整个网络，通过提高生成EEG样本的数量和质量进一步提高了EEG情感分类的性能。该项目的研究取得了一系列研究成果，对于实现高级人机交互与情感脑-机接口系统开发具有重要理论意义和学术价值。