基于子空间迁移学习的跨语种语音情感识别研究

Cross-lingual speech emotion recognition is a hot and cutting-edge research topic in speech emotion recognition field. In the research of cross-lingual speech emotion recognition, the speech emotion recognition model trained based on the training speech utterances would not be well suitable for recognizing the emotion categories of the testing speech utterances because of the significant differences between the emotional speech features extracted from the training speech utterances and the testing utterances. In this project, we will investigate the cross-lingual speech emotion recognition issue based on our preliminary research work of speech emotion recognition and propose a novel subspace transfer learning approach to deal with the cross-lingual speech emotion recognition problem. The research contents of this project consist of the following four major parts: (1) the method of emotional speech feature extraction and speech feature selection; (2) the method of transductive transfer learning based linear discriminative subspace learning as well as the corresponding optimization approach; (3) the method of transductive transfer learning based kernel discriminative subspace learning method as well as the corresponding optimization approach; (4) the method and algorithm of cross-lingual speech emotion recognition based on subspace transfer learning. This project will focus more on the unified model building of the subspace transfer learning method and its application to the cross-lingual speech emotion recognition. The proposed methods may also be applicable to the emotion recognition problem of other modalities.

跨语种语音情感识别研究是当前语音情感识别研究的热点和前沿课题。在跨语种语音情感识别研究中，由于从训练语音信号和从测试语音信号中提取的情感特征之间存在较大的差异，使得传统的基于训练数据得到的语音情感识别模型难以适用于测试样本数据。本项目旨在申请人原有语音情感识别与子空间学习研究基础上，深入开展跨语种语音情感识别研究，提出基于子空间迁移学习的跨语种语音情感识别理论与算法。所开展的研究主要包括以下内容（1）情感语音信号的特征提取与特征选择方法研究；（2）基于直推式迁移学习的判别子空间分析方法及其优化算法研究；（3）基于直推式迁移学习的核判别子空间分析方法及其优化算法研究；（4）基于子空间迁移学习的跨语种语音情感识别方法与算法研究。本项目侧重于子空间迁移学习方法统一模型构建及其在跨语种语音情感识别方面的运用，所提出的理论和算法思想可用于其他模态的情感识别研究中。

语音是人类情感交流的重要方式，通过对语音情感信号的分析与识别可感知人类的心理状态，在医疗、教育等领域具有重要的应用前景。因此，语音情感识别研究已成为当前情感计算研究的重要内容。在语音情感识别中，如何突破跨语种的情感识别所面临的挑战，即训练样本与测试样本来自不同语种语音情感数据库而造成特征向量空间分布不匹配的问题，已成为当前语音情感识别研究的热点和挑战性难题。本项目针对跨语种语音情感识别问题，开展基于子空间迁移学习的识别研究，主要研究成果包括：（1）提出了基于双稀疏学习的语音情感特征选择和识别方法；（2）提出了迁移子空间模型的统一理论框架和基于迁移子空间学习的特征选择方法；（3）基于语音和面部表情多线融合的情感识别方法。本项目实施过程中，在IEEE Transactions on Affective Computing、Speech Communication等国内外期刊和ACM MM、ICMI等会议上发表论文(含已录用)27篇，申请国家发明专利6项，其中已授权专利2项，获2018年国家技术发明二等奖。并分别获2019年度和2016年度ICMI国际情感识别大赛的冠军和季军。