基于RGBD序列的动态物体几何与纹理重建及其数据集建设

3D reconstruction of dynamic objects is a highly-focused problem in communities of both computer graphics and computer vision. It has wide applications in areas ranging from gaming, film-making, to simulation. The recently prevalent RGBD data acquisition techniques provide new methods for research on dynamic object reconstruction, which receives great attentions. However, reconstruction methods based on this kind of data are far from well-established, there are still many scientific problem to be studied. From viewpoints of both rgb-depth and space-time, this project explores the whole pipeline and the fundamental algorithms to robustly and efficiently reconstruct dynamic objects using RGBD sequence. First, this project will study how to improve the quality of the RGBD sequences using light-geometry-color relationship and temporal reduction. Second, this project will investigate the reconstruction framework and algorithms under the temporal data condition. In this part, the project will focus on the studies of how to deeply employ the information of both rgb-depth and objects' motion, so as to resolve the problems such as non-rigid registration with large motions and error-accumulation distribution. Third, this project will study how to reconstruct high-quality texture under the complex condition with objects’ motion, shading variation, and imperfect registration. Fourth, this project will acquire, compile, and publish to other researchers the associated datasets of both the developed algorithm and the reconstructed sequences of dynamic objects.

动态物体的三维重建是近年来计算机图形学、视觉研究领域的热点问题，它在游戏、影视、仿真等领域都有广泛应用。最近流行的RGBD数据获取技术，为动态物体重建研究提供了新的手段，受到广泛关注。然而目前基于此类数据的重建方法远未完善，许多科学问题亟待深入研究。本课题从图像-深度和空间-时间两个视角探索利用RGBD序列对动态物体进行鲁棒、高效重建的整体流程和基础算法。首先，研究利用光照-几何-颜色关联和时域冗余提高数据质量。其次，研究RGBD序列数据条件下的重建框架和算法。其中将着重研究如何深度利用图像-深度信息和物体运动信息，从而解决大幅度非刚体匹配、累积误差消除等难题。第三，着重研究在物体运动、光影变化、匹配欠精准等复杂条件下的高质量纹理重建。 第四，拟采集、整理、发布算法测试数据集和动态序列数据集，供其他研究人员使用。

动态物体的三维重建在游戏、影视、仿真等诸多领域有重要应用。本项目针对动态物体重建的关键科学问题展开探索，取得包括算法、系统、数据集在内的一系列研究成果。.针对RGBD数据含有较强噪声的问题，提出了一种基于稳定特征的RGBD数据对应与补全算法，有效提高低质量数据的匹配成功率；针对运动物体的形变特点，提出一种基于语义的非刚性物体稳定跟踪算法；对于非刚体重建结果，提出基于L0优化的点云去噪声算法框架；这些算法的提出，有效提高了非刚体重建过程的稳定性、方便性，有效提高重建结果的质量。.同时，本研究组提出一套基于非刚性配准的快速人体建模与体型估计算法，基于此算法的系统便于部署和使用，具有较强实用性；为丰富研究数据来源，本研究组构建了一套基于相机阵列的多视图纹理与几何数据采集与重建的软硬件系统，并基于此系统构建了高分辨率多视图人脸表情数据集。相关数据采集系统将为后续相关领域的研究提供数据采集的硬件环境，高分辨率多视图人脸表情数据集为后续进一步研究人脸精细几何特征与纹理提供了重要的数据基础。