基于微分几何和机器学习的立体视觉研究

Stereo vision is one of the important issues in the field of computer vision, which is widely applied in the fields of military, industry, medicine, etc. Using the multi-view images captured by two or more multi-view cameras, stereo vision automatically reconstructs the three-dimensional geometry of scenes. However, there are many limitations in existing methods when handling textureless regions, image noises, complex surfaces or information loss. How to improve the accuracy and completeness in stereo vision has become a problem that is exigent to be solved. This project aims at the research of the application of differential geometry and machine learning in stereo vision. Differential geometric invariants and the corresponding measurement methods of consistency will be researched based on the exploration in the local differential geometry features of various smooth manifolds. Besides, we will study the mutually beneficial relationship between machine learning (e.g., object identification, object classification, illumination ambient estimation). Furthermore, we will research the methods of feature extraction and parameters estimation based on online unsupervised learning, so as to realize the self-adaptive feature extraction and parameter settings without manual intervention. On the basis of these researches, a reasonable (higher-order) Markov random filed (MRF) model will be constructed. A fast algorithm for optimizing the model will be proposed and implemented. A practical multi-camera stereo vision system with high accuracy and good completeness will be finally built, in order to dramatically save human resource and improve the efficiency in real applications.

立体视觉是计算机视觉领域中的核心问题之一，即根据多视角摄像机拍摄的图像自动恢复场景的三维几何形状，广泛应用于军事、工业、医疗等领域。但现有方法在处理弱纹理区域、图像噪声、复杂曲面、信息缺失等方面都存在局限性，使得立体视觉在精度和完整性等方面的不足成为亟待解决的问题。本课题致力于微分几何和机器学习在立体视觉中的应用研究。通过探索光滑流形局部微分几何特性，研究微分几何不变量及其一致性度量方法；研究机器学习（如目标识别、目标分类与光照环境估计）与立体匹配的互利关系；研究基于在线无监督学习的特征提取和自适应参数估计，真正实现无人工干预的自适应的特征提取和参数设定。在此基础上，构建合理的（高阶）马尔可夫随机场模型，设计并实现模型最优化问题的快速求解算法；针对多视角视频序列，搭建具备精度高、完整性好、实用性强的多摄像机立体视觉系统，在实际应用中解放人力、提高效率。

作为立体视觉中最重要的一步，立体匹配的精度和效率在应用中显得尤为重要。但传统立体匹配方法在处理弱纹理区域、图像噪声、复杂曲面、信息缺失等方面都存在局限性。本课题致力于解决立体视觉在精度、完整性和计算效率等方面的不足，研究取得了重要进展。通过对弱纹理、复杂曲面、信息缺失和图像噪声引起的匹配歧义、误匹配、重建场景不完整的问题进行研究，提出了一种基于材质分类的双目匹配方法；利用大量标定数据和深度学习算法，研究并提出了一种基于机器学习的图像特征提取方法，实现了自适应提取欧氏空间可分的图像局部特征；同时，我们研究并提出了一种基于机器学习的自适应平滑约束方法，结合半全局匹配算法，实现了高精度的立体匹配；通过并行计算方法，利用GPU实现立体匹配的快速求解，将计算一个视差图的时间最终控制在1秒之内，为搭建应用系统奠定了良好的基础。本课题提出的方法在公开测试库Middlebury和KITTI上均取得了良好的测试精度和较高的效率，本课题研究的成果可用于科技、文化、教育、医疗、制造等非常广泛的领域，也有望应用于若干类似的领域，如基于主动成像的立体视觉、三维配准等。