宽基线条件下拼接式全景立体成像关键技术研究

In recent years, the rapid development of virtual reality put forwards higher demand for the performance of panoramic stereo imaging system. Focusing on the issue that the existing imaging systems could not support the roaming of viewers in the virtual scenes, we propose a wide-baseline multi-view solution for panoramic stereo imaging, which could support the roaming of viewers while preserving the realistic texture of scenes. The main researches are threefold: (1) scene modeling based on triangular facet approximation, originated from the idea of planar transformation and surface reconstruction, a scene modeling method based on triangular facet approximation is proposed, and the facet parameters can be acquired using the feature matching information between images; (2) occlusion handling based on spatial triangular facet model, the occlusions are detected using the geometric property that the projection of a occluded facet would be reversed in different views, then the foreground and background are segmented around the occluded areas to improve the precision and robustness of the scene model; (3) generation of panoramic stereo images in wide-baseline conditions, the projection relationships between the image planes of virtual and real cameras are derived from the transformation of facet parameters between different coordinates, and the final panoramic stereo images are rendered in units of facets.

近年来，虚拟现实领域的快速发展对全景立体成像系统的性能提出了更高的要求。针对现有成像系统不支持观察者在虚拟场景内自由移动的问题，本项目提出一种以宽基线多视图拼接的方式进行全景立体成像的解决方案，能够在保留场景真实纹理的同时，支持观察者的自由移动。研究内容包括以下三方面：（1）基于三角面片拟合的场景建模，借鉴平面投影变换和物体表面重构的思想，提出基于三角面片拟合的场景建模方法，并利用图像间的特征匹配信息求解面片参数；（2）基于空间三角面片模型的遮挡处理，利用遮挡会导致空间三角面片在不同视图的投影出现翻转的几何性质进行遮挡检测，并在出现遮挡的位置进行前后景分割，提高场景模型的精度和可靠性；（3）宽基线条件下全景立体图像生成，通过不同空间坐标系间面片参数的变换，得到虚拟相机与实体相机图像平面间的投影变换关系，最后以三角面片为单位渲染得到最终的全景立体图像。

本项目提出一种以宽基线多视图拼接的方式进行全景立体成像的解决方案。研究内容包括以下三方面：（1）基于三角面片拟合的场景建模，借鉴平面投影变换和物体表面重构的思想，提出基于三角面片拟合的场景建模方法，并利用图像间的特征匹配信息求解面片参数，求解所得面片参数较为准确地刻画了场景的三维结构；（2）基于空间三角面片模型的遮挡处理，通过对面片参数的分析，检测不同视图间遮挡，并在出现遮挡的位置进行前后景分割，有效提高了场景模型的精度和可靠性；（3）宽基线条件下全景立体图像生成，以三角面片为单位可在全景图像平面渲染得到重投影图像，通过引入基于径向基函数的图像配准方法，进一步降低了图像间细节纹理等复杂区域的配准误差，据此最终调整并融合得到高精度全景立体图像。