基于四元数整体空间点表达和三维Gabor降维处理的人体姿态估计

3D human pose estimation based on RGB-D images is hard to practical used. Because of the space expression is incomplete and in-diversity, and the proposed feature cannot describe the objective and human body parts accurately. All of them lead to bad result of region segmentation and labeled towards to human body. In addition, it is difficult to guarantee the efficiency and accuracy of algorithms. In order to solve these problems, we propose a new quaternion presentation of RGB-D space point, building holistic spatial representation based on quaternion and using the 3D Gabor filter to extract valuable feature for quaternion space point. Finally, the pose estimation was realized by using SVM. The main contents include: 1) based on the color image, the depth image was up-sampled and filled. A holistic spatial representation based on quaternion was proposed which realizing the integrity and diversity of the single point data. And the missing data in depth image was compensate by quaternion linear interpolation; 2) 3D Gabor filtering method was used to extract texture features of the 3D image. Quaternion vector with lower dimensional expressed the consistency of texture, color and depth was proposed which reducing the dimension and maintain the features extracted accurately; 3) based on quaternion feature vector, the neighborhood relations of space points are calculated. According to the space position relationship of human body parts and the 3D geometry structure, human body parts are accurately segmented and labeled. Then, we can get a complete human body region. Finally the SVM was used to accurately estimate human body posture. This project is a new way to explore whole space expression for RGB-D stereo camera, and has important theoretical significance and application value for the accuracy of human pose estimation and understanding.

当前基于RGB-D立体相机的人体姿态估计由于受数据缺失的影响，导致所提特征无法准确描述目标，进而对人体区域分割不准确，极大影响了姿态识别的准确性。为解决上述难题，本项目提出新的RGB-D空间点四元数表达，采用三维Gabor滤波提取空间纹理特征，并进行四元数一致性特征量表示。主要研究内容包括：1）提出四元数整体空间点模型，并对缺失数据进行补偿，保证单点数据信息的完整性和多样性；2）采用三维Gabor滤波方法提取空间纹理特征，并用较低维数的四元数向量对纹理、颜色和深度特征量进行一致性表示，实现对特征的准确提取；3）根据四元数特征向量进行空间点邻域关系计算，结合三维区域的空间几何特征，对人体各组成区域进行正确标注和分割，采用支持向量机实现对人体姿态的准确估计。本项目是对RGB-D相机的整体空间点表达方式的全新理论性探索，对正确理解和估计人体姿态具有重要的理论意义和应用价值。

1、项目的背景.本项目研究如何合理和有效的利用RGB-D图像中的六维信息进行人体姿态估计及行为识别。采用四元数对RGB-D立体相机的空间点进行整体表达，利用数据自身结构的关联性来解决原始采集数据的缺失和冗余问题。本项目的研究是对RGB-D立体相机四元数整体空间点表达方式的全新的理论性探索，对提高人体姿态估计及行为识别的准确性具有重要的理论意义和应用价值。.2、.主要研究内容.（1）提出了一种四元数Gabor彩色纹理特征提取方法，该方法可以很大程度的保留原图像的粗糙度，对比度和方向度等纹理特征，提取原图像的颜色信息，在特征表达方面优于传统Gabor方法和LBP方法。.（2）提出了一种基于四元数时空卷积神经网络的人体行为识别方法，采用四元数对彩色图像三通道进行整体特征提取，该方法提取的特征实现了低维一致性特征量表示，对准确的姿态估计和行为识别具有重要的意义。.（3）提出了一种基于时空骨骼特征及深度信念网络的人体行为识别方法，实现了基于3D骨骼点的时空动作特征设计，改进了特征编码及归一化方法，对时间序列特征和空间相对位置特征进行高层特征提取并完成了对时空特征数据的降维与动作分类，实验结果表明，本课题提出的方法能够有效提高人体行为识别的准确率。.3、.重要结果.实验结果表明，本项目提出的方法在通用数据集上实现了对人体姿态的准确估计及对人体行为的准确识别，具有较好的泛化能力和鲁棒性。.4、.关键数据及科学意义.本项目提出的基于四元数时空卷积神经网络的行为识别方法在Weizmann数据集上完成了基于3D CNN的行为识别实验，最后获得96.34%的准确率，比QST-CNN和3D CNN的准确率分别高出4%和6%左右；在UCF sports数据集上识别的准确率达到了93.2%。实验结果表明，采用提出的识别方法明显减弱了网络的过拟合现象。.提出的基于时空骨骼特征及深度信念网络的人体行为识别方法比同样适用DBN算法的DBN+HMM的正确率提高了11.4%，在MSRAction 3D数据库上的识别准确率为93.4%，优于采用单一特征的60%的识别准确率。表明提出的识别方法具有较好的泛化能力。