室内激光与视觉耦合的点云对象语义增强研究

Indoor digital map is the basis for indoor digital information service, it plays an important role in the description of indoor scene theme, spatial distribution, object structure, topology relationship and application of spatial analysis. Aiming at the problem of fast modeling of complex indoor scene, this proposal researches on building the coupling measurement theory model of laser scanning and visual system, proposing theory and methods to breakthroughs the challenge of object model extraction and enhancement from complex point cloud, trying to solve the problem of augmented reality, and finally validating the result, thereby providing technical support for indoor position, navigation, and location service. To do this, the basic theoretical framework of coupling measurement of laser scanning and visual system is built, to implement the problem of sensor integration and data acquisition for scene reconstruction and objectified mapping. The key technology of point cloud object-level semantic enhancement and labeling is developed both under constrained and unconstrained environment to form the optimizing and enhancement algorithm of point cloud from coupling measurement system. The key technology of fine scale object-entity semantic relevance and map entity embedding is developed to implement object-oriented indoor mapping and augmented reality. Based on the existing platform of integrated hardware, the prototype of data processing system is developed and tested under typical indoor scenes for validating the ability of measurement and mapping of point cloud from coupling system. Point cloud object-level semantic enhancement and labeling is developed both under constrained and unconstrained environment to form the optimizing and enhancement algorithm of point cloud from coupling measurement system. The key technology of fine scale object-entity semantic relevance and map entity embedding is developed to implement object-oriented indoor mapping and augmented reality. Based on the existing platform of integrated hardware, the prototype of data processing system is developed and tested under typical indoor scenes for validating the ability of measurement and mapping of point cloud from coupling system. Point cloud and labeled the objects both on conditionally and unconditionally environment. The core processing algorithm is developed for laser-visual tightly coupling observation system, which is highly compact and optimized. By studying semantic relations of target and matching technique of standardized component, both augmented-reality modeling and indoor scene objectized mapping are realised. Based on the previous platform of hardware integration, the prototype of data processing systemn is improved , and proposed overall evaluation criteria for mapping by empirical studies of typical indoor scene.

室内数字场景地图作为室内空间信息服务的基本载体，在场景主题、空间分布、部件结构、拓扑关系等信息精细表达和空间分析等应用中扮演着重要角色。本项目通过室内场景激光与视觉耦合测量机制的理论研究，突破复合点云对象模型的识别和增强、场景现实增强的技术难题，解决复杂室内场景的高质量快速重建问题，并对重建结果进行评价，为室内定位、导航和位置服务提供技术支撑。为此，本项目研究激光扫描与视觉耦合测量模式的基础理论框架，实现面向室内场景重建和对象化制图的传感器集成和数据获取；研究无条件／有条件约束的点云语义增强和点云环境模型对象标注的关键技术，形成激光与视觉耦合的点云模型增强和优化的核心算法；研究室内精细目标对象与标准部件之间的语义关联和匹配嵌入技术，实现室内场景对象化制图和现实增强；最后，基于已有硬件集成平台，构建耦合测量模式下的数据处理系统原型，通过典型室内场景的实证研究，评价复合点云室内制图的综合能力。

本课题研究围绕复杂室内场景的高质量重建问题，针对激光与视觉传感器联合标定方法、复合点云语义分割和标注、变化光照条件下的室内场景建模等多项关键技术展开研究，设计了耦合测量多传感器集成系统，给出了复合点云语义增强模型，实现了了室内场景高质量制图增强，验证了方法的有效性。.首先开展激光与视觉耦合测量系统的设计和开发，给出了多目视觉相机与激光雷达的多传感器集成与联合标定方案，建立了时空同步的耦合测量模式，实现了激光与视觉耦合测量的复合点云获取；在此基础上，研究室内场景复合点云语义增强方法，建立了上下文语义关联的点云语义标注框架，设计了顾及局部和全局上下文信息的深度神经网络，构建了压缩感知的复合点云张量空间约束，有效提高复合点云的标注与语义分割精度，增强了复合点云语义表达；然后研究了视觉与激光耦合的室内制图增强方法，根据耦合测量模式下复合点云的表现特点，给出了场景部件的无损体素化建模策略，面向室内光照变化、特征稀疏、部件相似等复杂场景，设计了空间-频域关联的匹配优化算法，构建了交叉验证的位姿-地图交互模型，提高了复杂环境下室内制图鲁棒性，实现了室内对象模型和场景现实制图的有效增强。最后给出了耦合测量复合点云的质量评价体系，从完整度、一致性、和几何精度等方面设计了归一化综合差异指标，评价了复合点云数据的建模效果，在大范围、特征稀疏的典型场景下优化了室内制图与定位结果，证明了视觉与激光耦合测量理论的有效性，实现理论与实证的衔接。.本课题研究验证了激光与视觉耦合测量理论在提升点云数据质量和构建高质量室内数据融合模型上的有效性和先进性，实现了从高质量复合点云获取、对象点云感知增强、高精度场景制图、场景完好性评价的完整链路，形成了耦合测量室内制图增强和优化的若干关键理论和核心算法，能够为复合点云在室内场景下的高效认知提供技术支撑，在建筑管理、室内导航、位置服务和应急响应等方面上提供工程指导。研究成果申请发明专利10项，所取得的系列创新成果已在《ISPRS Journal of Photogrammetry and Remote Sensing》、《Automation in Construction》、《Remote Sensing》等国际权威期刊上发表。