面向异类三维点云的语义迁移自动标注理论与方法研究

Semantic labeling of 3D point clouds is a key technology for information extraction. However, only trained on the point clouds collected by a single scanning device, labeling models based on traditional supervised learning methods cannot be directly applied to label heterogeneous 3D point cloud data obtained by different laser scanning devices. In addition, traditional labeling methods based on supervised learning rely on hand-crafted training data. Considering adaptability of the labeling model under the heterogeneous three-dimensional point cloud data and acquisition of the supervised information required at model training stage, the research exploits publicly annotated images and 3D model datasets as the source of supervised information, and focuses on three aspects: semantic labeling of point clouds via cross-dimensional label transfer, the similarity description between 3D mesh data and laser scanning point cloud data, and semantic labeling of heterogeneous point clouds via deep transfer learning. The research explores the theory and methods of automatic semantic labeling of heterogeneous 3D point clouds via semantics transfer. The main innovations include: (1) proposing a multi-constraints-based Markov Random model to transfer categorical labels among cross-dimensional data for automatic semantic labeling of point clouds; (2) proposing a weakly-supervised deep transfer learning model for labeling heterogeneous laser scanning point clouds. The output of this research will directly promote the application of semantic labeling of point cloud in various fields, such as virtual reality, digital city, autonomous driving, multimedia, etc.

语义标注三维点云是信息提取的关键技术。然而，传统基于监督学习的点云标注模型只在单一扫描设备所采集的数据上训练，难以适用于由不同三维激光扫描设备所获取的异类三维点云数据。另外，传统基于监督学习的点云标注方法仍依赖手工制作训练数据。本课题拟利用公开标注图像和三维模型数据集，针对异类三维点云数据下标注模型的适应性问题和标注模型训练所需的监督信息来源问题，开展基于跨维度标签传递的点云自动标注、三维网格模型与真实目标点云间相似性描述、基于深度迁移学习的异类点云语义标注三个方面的研究，探索面向异类三维点云的语义迁移自动标注理论与方法。主要创新点包括：（1）提出了基于多约束马尔可夫随机场的跨维度标签传递模型；（2）提出了基于弱监督深度迁移学习的异类点云标注模型。本课题的研究成果将直接推动三维点云语义标注技术在虚拟现实、数字城市、自动驾驶、多媒体等领域的应用。

随着激光扫描技术的飞速发展，不同的扫描设备在性能上有着显著的差异性。传统基于监督学习的点云标注模型只在单一扫描设备所采集的数据上训练，难以适用于由不同三维激光扫描设备所获取的异类三维点云数据。另外，传统基于监督学习的点云标注方法仍依赖手工制作训练数据。为了解决异类三维点云数据下标注模型的适应性问题和标注模型训练所需的监督信息来源问题，本课题开展了一系列的研究工作。主要研究内容包括（1）基于具有边界约束的马尔科夫图神经网络的点云目标分割，实现了弱监督条件下目标点云物体的准确分割。（2）基于多尺度特征表示与最大子图匹配的点云目标共同分割，实现了无监督条件下多个场景的公共点云目标的准确提取。（3）基于空间短语袋模型与联合分布适配模型的异类三维点云分类，在实现点云区域特征完整且鲁棒提取的同时，无监督地适配不同域的点云物体，进而实现异类点云的分类。（4）基于结合对齐网络与条件对抗适配网络的异类点云分类的域适应，实现了源域与目标域的类内点云目标的有效特征对齐，而且保证了类间点云目标的分类精度。本课题的研究成果将在三维目标识别与跟踪、智能交通系统等应用中发挥巨大作用。