基于多特征融合的自发表情识别研究

Along with the presentation of “Internet plus”, researches and applications of some disciplines such as artificial intelligence have to achieve a higher level. Facial expression recognition research is a foreshadowing of implementing the autonomous human-computer recognition and intelligent interaction. However, most of researches in internal and abroad are concentrated on posed expression. For the sake of serving the society with expression recognition research results better, this project will carry out research works around spontaneous expression database. In order to achieve the optimized facial expression recognition rate、noise reduction and strong robustness, several key issues and implementation methods in the spontaneous expression recognition would have been studied. The main works of this project are as follows: (1) A reliability research about the expression characteristics of spontaneous of static and dynamic facial features when human emotion changes. Research the integration of head movement features and static expression features. (2) Several major drawback of LBP algorithm are as follow: it is easy to be disturbed by noise; the amount of LBP feature dimensions is extremely high and LBP has a weak local discriminate ability. To overcoming these limitations, the project will research the optimized LBP facial expression recognition algorithm and sparse representation facial expression recognition. (3) In view of the complexity and diversity of expression change, this project will make much effort in researching the feature integration of different features based on sparse representation and making expression features fully characterized. Our purpose is to improve expression recognition rate and enhance the robustness.

随着“互联网+”概念的提出，使人工智能等研究与应用要达到更高的水平。人脸表情识别研究就是在为实现仿人类的人机自主识别和智能交互作铺垫，但是国内外大部分研究的是人为表情。为了更好地将表情识别研究成果应用和服务社会，本项目围绕自发表情数据库开展研究工作，对自发表情识别的关键技术和实现方法进行深入研究，以实现高识别率、低噪声、高鲁棒性的人脸表情识别为目的。主要包括：1）研究自发表情数据库的表情特点，充分利用人们情绪发生改变时脸部静态特征和动态特征的变化，研究头部运动特征和静态表情特征相融合。2）针对LBP易受噪声干扰、特征维数高和局部判别能力不强等问题，研究改进LBP和稀疏表示的人脸表情识别。3）针对表情变化的复杂性与多样性，研究不同特征在稀疏表示下的特征融合，使表情得到充分的表征，以提高表情识别率，并增强其鲁棒性。

自发表情相较于具有可欺骗性的人为表情，更能揭示一个人的真实情感，在国家安防、医疗等领域有巨大的应用潜力。自发表情诱导困难且具有强度低、幅度小、持续时间短的特性，易受到光照和头部偏移等因素影响，难以被认知。自发微表情的种类判别存在样本数量少、类别数据不均衡等问题。针对以上问题，本项目研究工作如下：1）研究特征融合和稀疏表示残差融合方法，使Curvelet特征和LBP/LPQ特征在稀疏表示下的残差不仅充分保持了各特征特性，而且有效地提取出各特征之间的差异性和相似性，使表情特征得到充分的表征。2）研究小数据样本深度迁移网络自发表情分类，为保留原始自发表情图片的特征，即使在小数据样本上也不使用数据增强技术。将样本置入不同的迁移网络模型中进行训练，并组合成同构网络，取得了目前最优的自发表情种类判别的平均准确率。3）研究混合时空平面局部二值模式的自发表情定位，该方法对头部偏移和光照强度都具有鲁棒性，通过精细匹配与特征值判定可以准确定位自发表情视频中自发表情的位序。4）研究基于对抗分解卷积网络的跨库自发微表情种类判别，能有效克服不同数据库中人种、肤色差异造成的影响，并且分解得到的自发微表情成分可以有效表达不同种类自发微表情的特征。5）研究基于过渡帧概念训练的微表情检测深度网络（MesNet），首次明确提出微表情过渡帧的概念，放大到监督学习领域，过渡帧概念对应于噪声标签的概念。MesNet检测微表情帧不依赖帧时序关系。因此MesNet不仅可以从微表情数据库完整视频中检测微表情帧，也可从给定的任意帧集合中检测微表情帧，还可判断给定的单独一帧是否为微表情帧。6）研究基于深度卷积神经网络和自编码器数据增强的微表情种类判别，创新性地扩充了微表情样本集合，让多人种、多数据库间的微表情识别不受限制，避免了在亚洲人微表情库上训练模型，在欧美人微表情库上测试性能较差的现象，反之亦然。