基于因子分析和姿态迁移的非正面表情识别算法研究

Most existing facial expression recognition methods only focus on the frontal or nearly frontal view facial images. The restraint imposed on head movements impedes the development of facial expression recognition. In this proposed project, we address the issues on the facial expression images with head rotations. Specifically, we investigate the factor analysis algorithm and transfer learning algorithm for non-frontal view facial expression recognition. On one hand, the factor analysis-based algorithm proposed for facial expression recognition under small head pose. By utilizing abrupt motion tracking, salient facial expression patches and factor analysis algorithms, the proposed method can extract the facial expression features from salient facial expression patches, and subsequently reduce the interference caused by different individual appearance, gender, age and other factors. On the other hand, we propose a second algorithm which employs a fashion of pose transfer to construct non-frontal view facial expression classification model to handle large head pose. This method is capable of extracting global and local facial expression features from human face images, and obtaining the facial expression information from different individuals with different head poses via feature fusion. Then based on the transfer learning theory, we can complete the knowledge transfer under the complex environment. Consequently, we improve the facial expression classification accuracy and generalization under head rotations. In summary, this project offers new research methods and techniques to achieve non-frontal view facial expression recognition, and contribute to the community of facial expression recognition.

人脸表情识别研究目前局限于正面或接近正面的人脸表情图像，限制了人类的头部运动，不利于人脸表情识别的智能化发展。本项目以头部偏转状态下的人脸表情图像为研究对象，分别从因子分析方法和姿态迁移方法入手，开展对非正面人脸表情识别的研究。针对小幅度头部偏转的人脸表情图像，提出了基于因子分析的非正面人脸表情识别算法。该方法以大位移目标跟踪、表情显著性区域检测和因子分析方法为基础，提取人脸表情显著性区域的特征，降低了不同个体的外貌、性别、年龄等因素对人脸表情识别的干扰。针对大幅度头部偏转的人脸表情图像，构建基于姿态迁移的非正面人脸表情分类模型。该方法提取面部图像的整体和局部表情特征，依靠特征融合方法获取不同人不同姿态的表情信息，进而通过迁移学习理论，完成复杂环境下的知识迁移，提高算法的识别率和推广性。本项目研究为实现非正面人脸表情识别提供了新的方法和技术手段，对促进人脸表情识别的实用化发展具有积极作用。

非正面人脸表情识别减少了对使用者头部偏转的限制，是人脸表情识别技术实用化发展的重要研究方向之一。本项目主要从三个方面开展非正面表情识别研究：适用于非正面人脸表情图像的表情显著性区域检测方法、依靠因子分析的非正面人脸表情识别方法，以及基于迁移学习的非正面人脸表情识别方法。在项目执行期间，共发表了学术论文15篇，其中SCI收录的论文9篇，EI收录的论文3篇，中文核心论文3篇；授权专利4项；培养硕士研究生9人，其中7人已毕业。通过项目的执行，我们在三个研究内容上都获得了重要进展。（1）在显著性区域检测方面，提出了一种非正面表情显著性区域检测方法；（2）针对小幅度偏转的表情图像，提出了基于因子分析的非正面表情识别方法。该方法在姿态、身份等众多变量中，寻找潜在的表情因子，降低了上述因素对表情识别的干扰；（3）针对大幅度偏转的人脸表情图像，构建了基于迁移学习的非正面表情识别方法。通过不同头部偏转条件下人脸表情图像的知识迁移，提高了分类器对大幅度偏转图像的表情识别率。上述研究成果，对非正面人脸表情识别的发展和应用具有重要的理论指导意义和应用价值。