非约束光照下人脸图像序列身份与情感信息分离识别研究

Identity recognition and emotion recognition are two important research directions of face analysis, but unfortunately the uneven illumination seriously restricts the development of them, so the successful separation and recognition of identity and emotion in face image sequence under unconstrained illumination are very important to the intelligent human-computer interaction. Firstly, this paper offers a different approach which analyzes the illumination distribution of a single frame face image by the illuminant direction estimation and then realizes the illumination normalization. Secondly, considering the individual difference and avoiding the negative influence of identity information for emotion recognition, the proposed approach extracts structure change information and texture change information of face image sequence whose illumination has been normalized. Thirdly, the proposed approach explores facial structure change rules and facial texture change rules respectively when the emotions happen, represents them as two mapping relationships, thus eliminates the negative influence of emotion information for identity recognition; furthermore, the proposed approach adjusts the recognition result by the candidate users’ emotion change rules which are from the pre-established and offline emotion change rule database. All the researches involved in this paper have both the value of theoretical innovation and practical application, which can actively promote the development of intelligent human-computer interaction.

身份识别与情感识别是当今面部分析的两个重要研究方向，而非均匀光照严重制约着两类识别的效果，实现非约束光照下人脸图像序列的身份与情感信息分离识别对实现人机智能交互具有十分重要的意义。本项目另辟蹊径，拟通过估计单帧人脸图像的光照方向，分析其光照分布，进而实现光照均衡；拟在考虑个体差异的前提下，提取光照处理后人脸图像序列的结构与纹理变化信息，从而规避身份信息对情感识别的影响；拟分别探寻情感变化时人脸结构与纹理的变化规律，并将此规律表征为情感发生前后的映射关系，从而消除情感信息对身份识别的影响，然后通过预先建立的情感变化规则库，利用疑似用户的个性情感变化规则修正身份识别结果。项目所涉及的研究具有理论创新和实际应用双重价值，能积极推动人机智能交互的发展。

人与人之间的交流可以通过多种途径进行，如语言，姿态，表情等。随着计算机技术的快速发展，人们希望人与机器之间的交互可以达到人与人交流的和谐程度。人脸情感与身份作为两种非常重要的信息传递方式，越来越受到研究者的重视。本项目通过计算机软硬件设备，探索人脸情感图像序列的自动截取方法，实现单幅人脸图像光照均匀与补偿，进而达到人脸表情与身份信息的分离及识别，使得人与机器之间的交互更加和谐，所做研究具体概括如下：（一）、对拍摄得到的人脸情感图像序列，即情感状态从平静过度到强度最大化，再回归到平静的变化过程，项目组提出了一种人脸情感序列截取方法；（二）、为解决人脸图像采集过程中遇到的光照不均问题，项目组利用光照方向估计结果，对传统的Retinex理论进行改进，进而实现单幅人脸图像的光照预处理，随后将人脸图像分割为阴影区域与非阴影区域并分别进行处理，达到光照补偿的目的；（三）针对光照处理后的人脸图像序列，计算各器官内部与各器官相互之间在二维平面上的结构变化信息，力求剔除两种信息的相互干扰，建立人脸图像的情感与身份信息分离模型，分别实现人脸情感与身份识别。近年来，项目所涉及的科学问题已成为国内外模式识别和人工智能领域的研究热点，能积极推动国家人工智能产业的发展。