三维场景深度的计算重建理论研究及其3DTV应用

High-quality depth acquisition has been a challenge for both academia and industry for a long time. Existing depth sensing methods and technologies have many shortcomings, to some extent, in terms of resolution, completeness, accuracy, and convenience, which heavily limits the development of related applications such as 3DTV and machine-vision. The project proposes the computational reconstruction theory and method for high quality depth map by constructing sparse representation and multi-prior model to integrate the advantages of different depth acquisition methods. The main research plan includes: 1) Investigate the advantages of different depth acquisition methods, set up degradation models for the depth sensing devices, and explore the fusion and correction methods to improve the accuracy of depth measurements; 2) Considering the intrinsic multi-model nature of depth signals, decompose depth signal into two components, and construction sparse representation for each component; 3) Construct the multi-prior regularized computational reconstruction theoretical model, and design fast and stable algorithms according to the structure of the proposed model. Finally, we explore a novel multi-view video coding scheme based on interested depth of field using the reconstructed high resolution depth map, then set up a 3DTV prototype system by integrating the proposed depth acquisition, reconstruction, and coding methods into a 3D video processing and display platform. The proposed depth reconstruction method can be extended to general signal reconstruction problems. Being fully developed, the proposed method can relief the lack of ground-truth data in the standardization of 3DTV format and coding, and has great potentials in content acquisition and generation of 3DTV.

高质量深度图的获取一直是学术界和工业界的共有挑战。现有深度获取方法和技术不同程度地存在分辨率、完整性、准确性、便捷性等方面的不足，严重制约了3DTV、机器视觉等领域的发展。本项目以融合多种深度获取机制的优势为切入点，构建深度信号的稀疏表示空间与多重先验模型，研究高质量深度计算重建理论与方法。主要内容包括：研究多元深度观测的降质建模，进而探索融合校正方法；分析深度和纹理的结构相关性，针对深度的多模型特性进行信号分解，构建各分量的稀疏表示空间；研究多重先验正则化的深度计算重建模型，并针对其结构特征设计高性能求解算法；探索基于感兴趣景深的多视点视频编码方法，整合本项目的采集、重建、编码方法，形成基于高质量深度的3DTV原型系统。本项目的深度重建理论与方法可以推广到更一般的信号重建问题，同时可以解除3DTV制式及编码标准化中真实深度数据匮乏的困境，在3DTV内容获取与生成方面具有很大的应用潜力。

高质量深度图的获取一直是学术界和工业界的共有挑战。现有深度获取技术难以对动态三维现实场景进行快捷精确的测量，存在分辨率、完整性、准确性、便捷性等方面的不足。为此，本项目融合多种深度获取方法的优势，提出多元深度观测的融合校正方法，揭示纹理和深度在结构上的相关特性，提出多重先验正则化的深度计算重建方法，并构建3DTV原型验证系统。本项目还在深度/图像预处理、三维运动与几何估计、立体图像分析等相关方向做了积极拓展研究，为深度计算重建算法提供预处理、算法应用等支撑。本项目取得丰硕成果有很大的潜力应用于立体影视、虚拟现实、辅助驾驶等产业。本项目在国际期刊发表SCI学术论文14篇，其中包括12篇IEEE Transaction长文，在图像处理领域知名国际学术会议上发表EI论文20篇（最佳论文奖2项）。申请专利25项，其中获授权国家发明专利7项。本项目的计算深度重建成果作为核心技术之一获得2016年天津市技术发明一等奖（“基于深度计算的舒适立体显示技术与应用”，负责人排名3）。本项目共培养四名博士生与22名硕士生，毕业学生均在学术界与工业界从事科研/研发工作，为国家信息技术产业发展输送一批优秀人才。