基于语义分割与理解的室外场景三维重建研究

3D reconstruction and modeling of outdoor scene, due to its vast applications in many areas, has long been an active topic in many research communities, from computer graphics, computer vision, photogrammetry, to remote sensing. Because automatic segmentation of the 3D scene has low accuracy rate and segmentation of the 3D scene requires a lot of human interactions, the large-scale outdoor scene is difficult to automatically reconstruct. 3D scene segmentation is regarded as an independent preprocess for the 3D scene reconstruction. Reconstruction process is lack of semantic information. In addition, the output models from existing approaches are usually a set of polygons with little semantic labeling. Editing models is difficult. The semantic information of 3D scene plays an important role in 3D scene reconstruction and its applications. So, in this proposal, we present an automatic approach to combine the bottom-up semantic segmentation based on energy minimization of higher order potentials and top-down semantic segmentation based on the context of outdoor scene for the high density, high noise and incomplete 3D point clouds of outdoor scene. At the same time, we use the semantic information to reconstruct the outdoor scene and refine the result of semantic segmentation during the reconstruction process. Then the system generates new training data set of outdoor scenes. The research on this project will largely promote the 3D reconstruction technology of outdoor scene and its application in city planning, navigation, surveillance, disaster simulation and computer games.

近年来，由于室外场景三维重建在许多领域的广泛应用价值，成为了计算机图形学、计算机视觉和遥感等多个领域的研究热点。三维场景自动分割准确率较低，需要大量人工交互，大尺度室外场景难以自动重建；场景重建将场景分割看作一个独立前置的过程，场景分割结果只是无语义的三维点云，在重建过程中缺乏语义信息的帮助，输出的模型通常只有一系列的网格，由于缺乏语义信息，难以编辑。三维场景语义信息在三维场景重建及三维场景的应用上扮演着重要的角色，所以，本项目结合自底向上的基于高阶能量优化的语义分割方法与自顶向下的基于三维场景上下文的语义分割方法，实现对高密度、高噪声、不完整的三维点云的语义分割；同时在室外场景三维重建过程中利用语义信息进行重建并反向强化语义分割结果，自动生成新的三维场景训练数据集。本项目的研究内容，将很大程度上促进室外场景重建技术的发展及其在城市规划、导航、监控、灾难模拟和计算机游戏等领域的应用。

近年来，由于室外场景三维重建在许多领域的广泛应用价值，成为了计算机图形学、计算机视觉和遥感等多个领域的研究热点。三维场景自动分割准确率较低，需要大量人工交互，大尺度室外场景难以自动重建；场景重建将场景分割看作一个独立前置的过程，在重建过程中缺乏语义信息的帮助，输出的模型通常只有一系列的网格，缺乏语义信息，难以编辑。三维场景语义信息在三维场景重建及三维场景的应用上扮演着重要的角色，所以，本项目提出了一种基于高阶类别相关先验知识的语义分割方法，从而实现对高密度、高噪声、不完整的三维点云的语义分割；并在室外场景三维重建过程中利用语义信息进行重建。同时，针对项目中遇到的问题和需求扩展研究了物体细类识别和非刚性物体三维重建。本项目的研究内容，将很大程度上促进室外场景重建技术的发展及其在虚拟现实、城市规划、导航、安全安防和计算机游戏等领域的应用。