结合数据驱动先验和图模型推断的深度图鲁棒增强技术研究

With the developments of information technology, the high-quality geometric information of real scenes receives extensive attentions in the academic and industrial communities. Since stereo matching cannot obtain robust depth maps in real time, the consumer-level depth sensors are widely used in many fields of people's living and industrial production. However, the quality of depth maps captured by such sensors is too low, so they need to be enhanced. Supported by the guidance of corresponding high-resolution color image, a deep convolutional neural network based on residual learning and dense skip is proposed to progressively reconstruct the high-frequency of the depth map. It is beneficial to restoring the detail under the large up-sampling factors; To improve the quality of depth edges, a loss function based on the description of edge quality is explicitly defined which is complemented by implicitly manifold learning via generative adversarial network; The integration of the data-driven prior and the prior defined by the graphic model which evaluates the edge inconsistency between the depth gradeient map and color gradient map is expected to mitigate the texture-copying artifacts and improve the robustness of depth map enhancement. This research can provide theoretical basis and technical guidance for depth map enhancement, meanwhile, a robust method for depth map enhancement is provided which establishs the solid foundation for Three-dimension Reconstruction, Virtual Reality and etc.

随着信息技术的进步，自然场景的高质量几何信息获取得到学术界和工业界的广泛关注。由于立体匹配算法无法实时获取鲁棒的深度图，大量消费级深度传感器被应用于人们生活和生产等方面。然而，使用深度传感器获取的深度图质量较低，需做质量增强。本项目拟以高分辨率纹理图的指导信息为支撑点，基于残差学习和稠密连接机制构建高效的深度卷积神经网络，逐步重构深度图的高频成分，解决大重建尺度下的细节恢复问题；结合基于深度图边界描述域显式定义的损失函数和基于生成对抗网隐式学习的数据流形提升深度图边界重建质量；结合数据驱动先验知识和嵌入纹理梯度图与深度梯度图之间边界不一致程度评估的图模型先验知识，抑制纹理拷贝赝像的产生，提升深度图增强算法的鲁棒性。本项目的研究可为深度图增强提供理论基础和技术指导，同时提供鲁棒的深度图增强算法，为三维重建、虚拟现实等技术的应用和发展奠定核心基础。

随着信息技术的进步，自然场景的高质量几何信息获取得到学术界和工业界的广泛关注。由于立体匹配算法无法实时获取鲁棒的深度图，大量消费级深度传感器被应用于人们生活和生产等方面。然而，使用深度传感器获取的深度图质量较低，需做质量增强。本项目拟从理论探索和算法设计方面针对高分辨率纹理图指导的深度图增强方法展开研究，为深度图增强提供理论依据和技术指导。研究方案以高分辨率纹理图的指导信息为支撑点，基于残差学习和稠密连接机制构建高效的深度卷积神经网络，充分挖掘纹理图的指导信息，逐步重构深度图的高频成分，解决大重建尺度下的细节恢复问题。同时，结合基于深度图边界描述域显式定义的损失函数和基于生成对抗网隐式学习的数据流形提升深度图边界重建质量。此外，本项目研究显式评估高分辨率纹理梯度图与深度梯度图之间的边界不一致程度，抑制纹理拷贝赝像的产生。最后，结合数据驱动方法和基于图模型定义的先验知识，提升深度图增强算法的鲁棒性。目前，本项目研究成果已发表十四篇SCI期刊论文、一篇EI会议论文、一篇中文核心期刊论文和出版专著一部。其中，以第一作者在IEEE Trans. Multim.和IEEE Trans. Circuits Syst. Video Technol.发表四篇；以通讯作者在本领域知名国际期刊IEEE RAL上发表三篇学术期刊论文；申请国家发明专利六项，三项已授权。