Time-of-Flight深度相机多径干扰问题的研究

With the development of modern technology, multimedia technology has entered the three-dimensional era from the two-dimensional era. Usually we need to add depth information to the traditional 2D image to get complete 3D information, and the depth information can be obtained by the depth camera. Time-of-Flight (ToF) technology is the current mainstream depth perception technology. It has the characteristics of high precision and high speed, and has been applied in large-scale industrialization. However, limited by the principle of ToF, ToF technology is still interfered by multipath interference. This interference can lead to erroneous depth information, which in turn causes distortion of the three-dimensional reconstruction based on depth information. This project intends to establish a simulation model for multipath interference of ToF depth camera, and design a ToF depth camera multipath interference suppression algorithm based on deep learning. The simulation system is built on the basis of the model and algorithm, and the performance of the algorithm is tested in the real scene. This project hopes to make breakthroughs in the key technologies to suppress multipath interference of ToF cameras, thereby improving the accuracy of ToF depth cameras and promoting the application of ToF cameras in accurate 3D reconstruction.

随着时代的发展，多媒体信息技术已经从二维时代走入了三维时代。通常我们需要在传统的二维图像的基础上增加深度信息来获得完整的三维信息，而深度信息可以通过深度相机来获取。时间飞行(Time-of-Flight, ToF)技术是目前主流的深度感知技术。它有着高精度和高速度的特点，并且已经得到了大规模产业化应用。但是，受限于ToF的原理，ToF技术仍然受到多径传播的干扰。这种干扰会导致错误的深度信息，进而使得以深度信息为基础的三维重建失真。本项目拟建立ToF深度相机的多径干扰仿真平台，设计一种基于深度学习的ToF深度相机多径干扰抑制算法。在模型和算法的基础上构建仿真系统，并在真实场景下试验算法性能。本项目希望在抑制ToF相机多径干扰的关键技术方面取得突破，从而提高ToF深度相机的精度，促进ToF相机在精准三维重建方面的应用。

随着时代的发展，多媒体信息技术已经从二维时代走入了三维时代。通常我们需要在传统的二维图像的基础上增加深度信息来获得完整的三维信息，而深度信息可以通过深度相机来获取。时间飞行(Time-of-Flight, ToF)技术是目前主流的深度感知技术。它有着高精度和高速度的特点，并且已经得到了大规模产业化应用。但是，受限于ToF的原理，ToF技术仍然受到多径传播的干扰。这种干扰会导致错误的深度信息，进而使得以深度信息为基础的三维重建失真。本项目建立ToF深度相机的多径干扰仿真平台，设计三种基于深度学习的ToF深度相机多径干扰抑制算法：基于空间层次解构的ToF深度图像多径干扰抑制算法，基于展开式建模的ToF深度图像多径干扰抑制算法，基于神经场景表达的ToF深度图像多径干扰抑制算法。这些算法有助于ToF深度图像质量的提升，比如在TFT3D 和 FLAT 数据集上，我们设计的基于空间层次解构的算法在测试集上的评估实验里分别将MAE降低到了1.19厘米和0.46 厘米。在知名数据集Cornell-Box上，我们设计的基于展开式建模的方法已经将MAE值降低到2厘米以下，超越了同期的各种方法。这些方法都以文章或专利的形式发表或公开。本项研究拥有比较好的应用前景，比如手机深度摄影的质量提升，自动驾驶等等。