基于机器学习与单目视觉的个性化人体建模技术研究

Reconstruction of 3D human models has recently gained tremendous interests as its widespread use in domains such as computer games, online virtual try-on and ergonomics. Scanning cost, speed and usability are important challenges for the state-of-art methods. In this project, we aims for the digital avatar application and takes research on 3D human model reconstruction based on monocular video and machine learning algorithm. This method has the advantage of low cost since it does not require any additional 3D scanning equipment. The scanning speed is also guaranteed which satisfies the requirements of the real-time interaction in virtual reality application. Besides, the reconstructed model has high quality which can be directly used in the online applications such as virtual try-on without any modification. Research contains the following specific lines of investigation: (1) explore a pose-invariant statistical model based on Laplace differential coordinates and statistical analysis, which can separate details from the basic model; (2) explore the feature representation method which combines the self-learned feature from convolutional neural networks with shape indexed feature; (3) explore the 3D human model reconstruction method from single or multiple still images based on the cascaded classification-regression forest and the proposed statistical model; (4) explore non-rigid mesh align algorithms under different views to further improve the reconstruction quality. This project takes a new try with the machine learning in the research of 3D human model reconstruction. The project outcome will have a wide range of applications in games, virtual reality, experience-based tourism and other domains.

三维人体模型由于在游戏、虚拟试衣、人机工程等领域中的广泛应用而成为近年的研究热点。本项目面向数字虚拟化身等精度要求不高的虚拟现实应用，拟突破三维人体重建过程中成本、速度以及易用性方面的制约，结合机器学习算法，研究基于单目视觉的的三维人体模型快速建模技术。该方法无需额外的三维扫描设备，成本低廉，简单易用，重建速度快，重建模型质量高，无需后期处理即可应用于虚拟试衣等在线应用。研究内容包括：基于基本几何模型与细节模型分离的思想，利用Laplace微分坐标构建与姿态无关的参数化人体模型；研究利用卷积神经网络自学习特征与形状索引特征相结合的特征表示方法；基于级联分类-回归森林的机器学习方法，研究基于单幅或多幅静止图像的三维人体模型重建方法；研究多视角下非刚性人体模型的对齐与融合的方法，以提高三维重建的质量。本项目为三维人体重建的研究提供了新思路，在游戏、虚拟现实以及体验式旅游等领域中有广泛的应用。

三维人体模型由于在游戏、虚拟试衣、人机工程等领域中的广泛应用而成为近年的研究热点。本项目面向数字虚拟化身等精度要求不高的虚拟现实应用，研究了基于单目视觉与深度学习的三维人体重建的关键技术，设计与实现了一个基于深度学习与单目视觉的三维人体重建系统。主要的研究成果包括：（1）分析与比较了常用的三维人体模型的参数化表达，设计了一种改进的SMPL参数化人体模型；（2）基于改进的SMPL参数化人体模型，提出了一种基于生成对抗网络模型的三维人体重建方法；（3）基于分而治之的思想，提出了一种基于单目视频的动作识别方法。本项目已申请发明专利1项，软件著作权1项，发表学术论文4篇，培养学生6名，已达到项目的预期成果要求。本项目为三维人体重建的研究提供了新思路，在游戏、虚拟现实以及 体验式旅游等领域中有广泛的应用。