基于稀疏表达理论和RGBD图像的人脸表情识别

Currently, one of the most difficult problems needed to be solved in the field of facial expression recognition is how to build a recognition method robust to occlusions, corruptions and various illuminations. Sparse representation based classification (SRC) method is proposed by simulating biological visual system, and previous studies have demonstrated it has a certain robustness to occlusions and corruptions. This project intends to extract the two-dimensional features and three-dimensional features from RGBD images, and uses these features combined with SRC to generate different recognition methods, and analyze the fusion strategy of different classification methods with different features. This applicant firstly studies the extraction methods of two-dimensional facial expression features based on largest-differences- between-classes images and three-dimensional expression features based on depth images, and then use SRC combined with those features to classify facial expressions and acquire numerous identify results, and analyze the fusion strategy of multiple classifiers based on Bayesian theory to fuse those results because of the complementarity between different results of recognition methods, and finally test the robustness of proposed facial expression recognition method on the images with occlusions, corruptions and various illuminations. This study will open up new ways for developing facial expression recognition technology; a new attempt to recognize facial expressions by fusing two-dimensional and three-dimensional image features, and aims to provide an expression recognition framework with better versatility and applicability.

当前表情识别领域亟待解决的难点问题之一为如何构建一种能对遮挡、噪声和光照不均等异常情况鲁棒的识别方法。基于稀疏表达理论的分类方法是通过模拟生物视觉系统提出的，前期的研究证明了其对遮挡和噪声具有一定的鲁棒性。本项目拟从RGBD图像中分别提取二维特征和三维特征，并结合基于稀疏表达理论的分类器，研究将二、三维图像特征融合识别人脸表情的策略。本项目首先研究了基于最大类间差别图像的二维表情特征和基于深度图像的三维表情特征的提取方法，接着将提取的特征构成字典分别结合基于稀疏表达理论的分类器得到多种识别结果，然后利用基于不同特征分类方法识别结果之间的互补性，分析基于贝叶斯理论的多分类器决策级融合规则，最后在含遮挡、噪声或光照不均的图像上验证方法的鲁棒性。本研究将为表情识别技术的研发开辟新途径，是一次融合二、三维图像特征进行表情识别的全新尝试，旨在提供一个通用性和适用性更好的表情识别方法框架。

本项目旨在研究提出一种基于二、三维特征融合和稀疏表达理论的人脸表情识别方法，以期实现一种能够对遮挡、噪声和光照等鲁棒的方法，并且通过结合协作稀疏表达，使提出的方法具备一定的实时性。针对以上项目背景，本课题重点针对五个方面展开了研究：1）基于稀疏表达理论的人脸表情识别框架研究；2）二维表情图像的特征字典构建方法研究；3）三维表情模型的特征字典构建方法研究；4）基于多特征决策级融合策略的研究；5）基于Kinect的RGBD人脸表情数据库构建。在研究现有二维、三维特征与SRC结合的人脸表情识别方法基础上提出了两种新的基于稀疏表达理论的人脸表情识别方法。分别为：1）基于“标称—特性”双字典和多特征融合的人脸表情识别方法，2）基于二、三维特征融合与协作稀疏表达的人脸表情识别方法。经过实验，第一种方法相较于现有的二维人脸表情图像的特征提取方法能够取得更好的识别准确率，在CK数据库上达到94.56%，在3D-BUFE数据库上达到90.43%，第二种方法则是提供了一个将二维、三维特征信息融合识别的方法框架，通过在3D-BUFE数据库和FE-Kinect数据库上的实验验证，三维信息主要在二维图像受到中等以上光照影响时会给予互补，使得能够对光照鲁棒。本课题的研究成果具有一定的应用前景，可以广泛应用于现代远程教育网路、安全监控领域、医疗领域等。