多视点协同的结构光深度值获取关键问题研究

Owing to the advantages in accuracy, efficiency and price, structured light technique is a good choice of acquiring multi-view depth values and obtaining complete geometry information of 3D scenes. However, the increasing viewpoints also bring challenges to structured light techniques. For example, the problems of inter-view multiplexing pattern design, multi-view phase unwrapping and the phase errors are still to be solved. . We will carry out, in this proposal, research on the aforementioned problems. To be specific, based on the very fundamental principle of stereo vision and the characteristics of multi-view structured light, we will propose a set of multi-frequency-modulated projection patterns, which will solve the problem of inter-view interference and is the foundation of multi-view multiplexing and phase unwrapping. After that, we will have a deep insight into the multi-view structured light system and reveal the correlations, including multi-view geometry correspondence, multi-view texture correspondence and multi-frequency phase unwrapping principle. Based on these constraints, a phase unwrapping model and method will be proposed to compute multi-view unwrapped phases at once. Then, considering different kinds of information, i.e. the captured patterns, unwrapped phases, phase reliabilities and the phase smooth property, we will develop a joint phase optimization model and algorithm with Bayes' theorem. This model will generate the optimal phases by maxing the phase probability, which will improve the phase accuracy. Last but not least, the enhanced phase will be converted to depth. The proposal will enrich the techniques of multi-view depth acquisition, where the aim of complete, dense and real-time depth acquisition will be realized, and also facilitate a range of applications, such as virtual reality/ augment reality, free viewpoint video and 3D printing.

结构光深度值获取在精度、速度和成本上具有综合优势，可实现多视点深度值采集，更完整地获取场景几何信息。然而，视点的增加带来了新挑战，如视点间干扰、多视点相位解缠、相位误差等问题。. 本项目围绕以上问题展开研究。首先依据立体视觉理论和多视点结构光的特性，拟采用多频率调制方法建立投影模板，为解决视点间干扰、实现视点间复用和相位解缠奠定基础；然后通过研究系统内视点间的相关性，包括几何一致性、纹理一致性和多频率解缠约束，拟提出多视点相位解缠模型，以同步求取多视点解缠相位；进而在贝叶斯框架下，综合考虑采集模板、相位、相位可靠性及相位平滑性约束，拟提出概率最大化的相位优化模型及算法，以提高解缠相位精度；最后将优化所得相位转换为深度值。本项目拟进一步丰富多视点深度值获取的研究，达到完整、精确、实时地获取多视点深度值的目标，并促进其在虚拟现实/增强现实、自由视点视频和3D打印等中的应用。

结构光技术是实现场景深度值测量和三维重建的重要方式，然而单个结构光单元的视野有限，只能覆盖部分场景。本项目通过多视点结构光单元进行深度值采集，以获取更完整的场景几何信息。项目针对多视点结构光系统的一系列关键问题展开了研究，主要包括投影模板设计、混叠模板处理、相位解包裹、深度值优化、并扩展到相位编码的结构化光场的深度值测量。在模板设计方面，本项目提出了互质多频率条带模板和混叠模板，为相位解包裹和空域立体匹配提供了基础。在混叠模板处理方面，项目提出了基于卷积神经网络的分离算法以分离不同来源的模板，同时也提出了基于双目一致性的匹配算法直接利用混叠模板求解深度。在相位解包裹方面，项目从实时性、鲁棒性、数据驱动三个出发点提出了解包裹算法。从实时性考虑，提出了基于互质频率和几何平滑约束的方法；从鲁棒性考虑提出了基于多锚点的扫描线解包裹方法；从数据驱动考虑，从采集模板和包裹相位中提取特征进行训练，直接预测正弦波周期或者形变周期。以上方法提供了快速、准确、鲁棒地获取绝对相位和形变相位的方法。最后，针对获取的深度值存在空洞的问题，我们提出了基于任意形状样例块和多模混合相似度的修复优化方法。此外，针对视觉系统将从多视点向光场扩展这一趋势，我们研究了经典光场下基于极平面图像的多尺度渐进深度值估计；并将结构光和光场结合，研究了相位编码结构化光场的深度值估计，提出了基于极平面编码一致性的凸优化快速算法。通过以上研究，我们贯穿了多视点结构光系统深度值测量的各个环节，提出了模板设计、模板处理、相位解包裹、深度值优化等一系列方法，已发表和在审论文12篇，申请专利2项。项目研究成果可实现完整、精确、高效、鲁棒地获取多视点深度值的目标，可促进消费娱乐中的虚拟现实/增强现实、工业制造中的3D打印等、以及文物保护中的三维重现等应用。