基于深度学习与知识图谱相结合的水下三维场景感知研究

The researches and explorations in oceans have drawn attentions from more and more counties around the world in recent years. For example, search and rescue in seawaters, underwater archaeology, marine aquaculture, or marine military among others. The data acquisition and scene perception are playing key roles in those underwater areas all the time. Compared to the traditional 2D data, the three-dimensional perception for underwater data and underwater scene provides better and richer information, which can thus enhance the accuracy and efficiency for the tasks in the field of underwater explorations. Up to now, the existing solutions for underwater three-dimensional perceptions are mostly based on the utilization of dedicated 3D sensors, such as sonar. However, apart from the high system cost, those traditional solutions suffer from a range of limitations. For instance, the solutions based on sonar are only suitable for mid/long range sensing, and lacking the perception for colors. .To this end, this project proposal presents a research for deep learning and knowledge graph augmented three-dimensional underwater data acquisition and scene perception. It adopts the platform from a ROV. Based on the sole utilization of the optical camera as the data sensor to provide the data input, the proposed research combines the deep learning and knowledge graph to achieve underwater 3D perception and 3D data acquisition. The proposed research is based on the computer vision algorithms, including the underwater visual data enhancement based on a generative adversarial network, the underwater simultaneously localization and mapping based on the depth estimation through deep learning, and the loop-closure detection for underwater three-dimensional scene based on a knowledge graph. The proposed project presents a research to utilize the deep learning architecture for underwater 3D perception based on visual data. The anticipated outcomes of the project can be widely deployed in almost all fields for underwater explorations, and will lead to more following-up research in the similar fields. It can also further push the oceanology development of our nation to a higher level. The proposed research thus has significant scientific values and can be beneficial for a wider range of stakeholders in real-world scenarios.

近年来，海洋领域的研究和探索受到越来越多国家的关注，例如海洋搜救、海洋考古、水产养殖、海洋军事等领域。海洋数据的获取及水下环境的感知，在海洋研究和探索中起到至关重要的作用。相比于传统二维数据，水下数据及场景的三维感知，能提供更丰富的水下信息，提高海洋研究的精度和效率。目前国内外已有的水下三维感知大都基于专用三维数据采集设备，包括声纳、激光成像等。除价格昂贵外，这些传统方案存在一定的局限性，例如声纳只适合中长距离感知，且缺乏色彩信息。.因此，本项目以水下航行器为平台，使用光学相机为感知设备，基于深度学习和知识网络，对水下三维数据获取及三维场景感知开展研究。其中涉及基于深度学习的水下视觉数据增强算法、基于水下视觉数据景深预测的同步定位及构图算法，以及基于知识图谱的水下场景闭环检测算法。研究成果可广泛应用于海洋科学研究和探索的各个领域，进一步推动我国海洋科学发展，具有重要的科研意义和应用价值。

近年来，海洋领域的研究和探索受到越来越多国家的关注，例如海洋搜救、海洋考古、水产养殖、海洋军事等领域。海洋数据的获取及水下环境的感知，在海洋研究和探索中起到至关重要的作用。相比于传统二维数据，水下数据及场景的三维感知，能提供更丰富的水下信息，提高海洋研究的精度和效率。目前国内外已有的水下三维感知大都基于专用三维数据采集设备，例如声纳、激光雷达成像等。除价格昂贵外，这些传统方案存在一定的局限性，例如声纳只适合中长距离感知，且缺乏色彩信息。..因此，本项目以水下航行器为平台，使用光学相机为感知设备，基于深度学习和图网络结构，对水下三维数据获取及三维场景感知开展研究。其中涉及基于深度学习的水下视觉数据增强算法、基于水下视觉数据景深预测的同步定位及构图算法，以及基于图像一致性的水下场景闭环检测算法。经过项目期的研究推进，截止本项目结题，在基于深度学习水下图像增强及水下特征感知增强、基于深度学习的单帧RGB图像景深估计、基于深度学习的相机位姿估计、水下SLAM算法、水下三维重建算法、基于图像一致性的闭环检测算法领域，均形成重要成果和算法原型，形成标注本项目资助的学术论文8篇（其中两篇为已录用待发表）、专利一项，并迭代设计和构建了三代水下ROV实验平台，在可控水池和相关实际海域进行水下真实场景的实验，并得到理想的实验结果，实验精度和稳定性都较为可观。..基于本项目的研究，预期与相关部门合作开展一系列水下观测任务，包括南海珊瑚礁生长监测、灵山岛牡蛎礁生成监控、基于ROV平台的船壳清洁作业等。基于水下视觉的方案，研究成果可广泛应用于海洋科学研究和探索的各个领域，包括基于水下视觉的深度学习算法模型、水下ROV控制和人机交互等学术领域，以及水下勘探、水下作业等工程领域等。本项目的成果和扩展研究，可进一步推动我国海洋科学发展，具有重要的科研意义和应用价值。