基于立体视觉与深度学习的地外天体巡视器自主环境感知方法

In the surface rover exploration missions of extraterrestrial celestials, because of the complex unknown environment and the long distance between celestials and earth, the rover must have high autonomous ability of environment perception. Currently, most rovers are controlled through ground teleoperation, which is lack of deep understanding and can’t adapt to the real-time and safety requirements of the probe. This project uses the stereo vision sensors, and puts forward an autonomous environment perception method combined with stereo vision and deep learning, which will mainly discuss several aspects of the scientific issues and key technologies. Firstly, a fast stereo matching and 3D reconstruction method based on region segmentation is proposed. Secondly, according to the results of 3D reconstruction, the obstacles such as rocks and craters are extracted, and then the image is divided into flat area and obstacle area. Thirdly, the depth learning model based on transfer learning is studied by using the obstacles in the on-line learning, and the obstacles in the middle or remote distance can be detected based on the learning model. Lastly, using the on-line vibration and its corresponding visual image information, a depth learning model based on small samples is established, with which the traversability of the flat terrain is analyzed. Through the organization and implementation of the project, it is expected that an effective approach of autonomous environment perception is put forward, which will provide key technical support for future deep space exploration research program.

在地外天体表面巡视探测任务中，复杂未知的环境、与地球间遥远的距离，决定了巡视器必须具有自主环境感知能力。而目前主要通过地面遥操作控制巡视器的活动，缺乏对环境的自主理解，难以满足巡视器的实时性和安全性要求。本课题立足巡视器上的立体视觉传感器，提出结合立体视觉与深度学习的地外天体巡视器自主环境感知方法。将重点探讨以下几个方面的科学问题和关键技术：首先，提出基于区域分割的快速立体匹配与三维重建方法；其次，根据三维重建结果，提取岩石、坑等障碍物，将图像中需要识别的目标分为平坦区域和障碍物；然后，利用在线获得的障碍物建立基于迁移学习的深度学习模型，并对中远距离的障碍物进行检测；最后，利用在线振动及其对应的图像信息建立基于小样本的深度学习模型，根据该模型判断平坦区域的可通行性。通过本课题的组织和实施，预期提出地外天体巡视器自主环境感知的有效途径，为未来深空探测研究计划提供关键技术支撑。

本报告立足地外天体巡视器自主环境感知技术研究，提出结合立体视觉与深度学习的地外天体巡视器自主环境感知方法。重点探讨以下几个方面的科学问题和关键技术：首先，研究了一种高效的半全局快速立体匹配方法，该方法提出了了基于3D平面拟合、自适应惩罚系数、从粗到精的视差约束等策略；其次，提出了基于超像素区域分割的立体匹配方法，该方法对于弱纹理区域的立体匹配也能够得到较好的效果；接下来，根据三维重建结果，提取岩石、坑等障碍物，将图像中需要识别的目标分为可通行区域和障碍物，建立了基于火星数据的样本库、第一套基于嫦娥三号数据的样本库；然后，利用在线获得的障碍物建立基于迁移学习的深度学习模型，并对中远距离的障碍物进行检测；最后，建立基于小样本的深度学习模型，提出了基于深度学习的天体表面地形识别分类模型与可通行性研究。通过本课题的组织和实施，提出地外天体巡视器自主环境感知的有效途径，可为未来深空探测研究计划提供关键技术支撑。