基于计算光场的复杂场景三维人脸重建关键技术研究

Computational light-field imaging has been one significant technological renovation. High-dimensional light fields provide novel opportunities for computer vision and biometrics, however the study of biometrics based on light-field (LF) imaging is still in the primary stage. The project aims to carry on the research of 3D face reconstruction based on LF imaging under uncontrolled environment ("In-the-Wild"). We innovatively introduce the theory and method of LF imaging to 3D face reconstruction for data acquisition. By combining traditional SV and LF imaging technology, we study on the construction of a novel LF 3D face database, the theory of multi-baseline LF-SV face imaging, and the computational algorithms for LF 3D face reconstruction. The unique characteristics of 4D LF data is absolutely exploited to solve the challenging tasks of 3D face research based on traditional imaging, e.g. lighting, pose, expression. The goal is to establish the 3D face imaging and reconstruction system with high robustness, reliability, precision and efficiency to interaction, application and environment. Our research provides powerful supports of the data, theory, and algorithm for 3D face recognition. It opens up new research topics of face recognition in the era of LF imaging. The project will definitely promote the collaborative innovation and development of biometrics, optical engineering, computational photography, computer vision, etc.

计算光场成像相对传统光学成像是一次重大技术革新，高维光场信息为计算机视觉和生物特征识别的学科创新带来了新机遇，但基于计算光场成像的生物特征识别研究还处在初期探索阶段。本项目旨在研究基于计算光场的复杂场景三维人脸重建，从数据获取源头创新，将计算光场成像理论和技术引入三维人脸重建研究，融合传统立体视觉与光场成像技术，研究光场三维人脸数据库建设、多基线光场立体视觉人脸成像理论、光场三维人脸重建计算算法，挖掘高维光场信息解决基于传统成像的三维人脸研究面临的难题（如光照、姿态、表情），创建对用户交互、应用场景、成像环境高鲁棒性、高稳定性、高精度、高效率的光场三维人脸成像和重建系统，为三维人脸识别研究提供有力的数据、理论和算法支撑，开辟人脸识别学科在计算光场成像时代的新方向，推动生物特征识别、光学工程、计算摄影、计算机视觉等学科的协同创新与发展。

本项目旨在挖掘计算光场成像技术在生物特征识别研究（虹膜、人脸）中发挥的作用和意义，从成像系统设计、成像过程分析、计算算法创新三个维度展开研究，融合深度学习模型框架、传统立体视觉技术、光场成像技术，提出了双目光场成像理论、遮挡鲁棒的深度感知、高保真视角合成、二维/三维人脸关键点定位、三维人体形状及姿态估计、三维人体模型重建、虹膜活体检测、人脸活体检测、三维人脸识别等算法，一定程度上解决了基于传统成像技术的生物特征成像、表达、识别的难点与挑战，是一个完整的系统性研究工作。基于本项目，共发表学术论文11篇、申请发明专利6项，自主研制虹膜、人脸成像系统，建设光场人脸数据库、虹膜数据库。本项目在成像系统和计算算法上的创新，促进了计算成像、生物特征识别、光学工程、计算机视觉等学科的协同发展与应用。