大视角三维光场显示中基于无监督深度学习的密集视点生成方法研究

Three-dimensional (3D) light field display, as one of the potential directions of future display technology, has still attracted many researchers' attention, because of its large view-angle, high resolution and real-time rendering. However, 3D light field display still faces many challenges. One of them is the number of viewpoint is insufficient for the light field display..Here, the research of multi-view synthesis for light field display based on unsupervised deep learning is present. The principle of minimum discrete sample of light field is established to achieve sparse views collection. The principle of continuous light field restoration is proposed to achieve dense views synthesis. A self-manufactured light field display is used to verify the displaying result. Based on the performance, parameters of view capture and view synthesis are adjusted and optimized. .The method of unsupervised deep learning is used to synthesize the virtual dense light field viewpoints with sparse light field viewpoints. Compared with the supervised network, it does not decrease the quality of synthesized result, without requiring a large number of training targets, and the virtual location can be flexibly adjusted. The realization of this research will greatly simplify the construction of light field capture, and lay a foundation for the wide application of light field display.

三维光场显示，因其视角大、分辨率高等特点，受到众多研究人员的青睐。但是，三维光场显示技术依然面临着很多挑战，其中之一就是光场显示中视点数量采集不足的问题。. 因此，本课题将开展大视角三维光场显示中基于无监督深度学习的密集视点生成方法研究，采集稀疏光场视点，合成密集虚拟视点，验证显示三维光场效果。构建大视角光场的最小化离散抽样机理，实现大视角三维光场的稀疏视点采集；提出大视角连续光场的恢复原理，实现大视角三维光场的密集视点生成；利用自主研发的三维光场设备进行显示验证；评估显示结果，优化调整采集及合成中多项参数。.本课题利用无监督深度学习的方法，将采集的稀疏光场视点合成为虚拟的密集光场视点。相比于有监督网络，可以在保证视点质量的前提下，不需要大量的虚拟视点作为训练目标，并且生成的虚拟位置能够灵活调整。本课题的实现将极大的简化光场采集的难度，为光场显示的广泛应用打下基础。

本项目为了突破三维光场显示中采集视点数量不足的困难，从视点内容相关性出发，研究了无监督的视点生成算法，在采集少量的视点前提下就可以拥有高质量密集生成的视点，满足三维光场显示的需求。为了生成虚拟视点的信息，我们从相邻视点特征相似和视觉几何强相关这个角度出发，探索了无监督虚拟视点合成方法，大幅度提升了虚拟视点的生成质量，满足了三维光场显示的需求。为了提升视点合成质量，我们充分研究了水平视差对于两视点生成的影响，利用卷积神经网络的拟合功能，有效提升了视点匹配精度，错误单个像素的占比指标也稳定在10%以下，视点生成质量稳定在PSNR28dB以上；另外，针对任意姿态摆放的多视点摄像头采集方案，在考虑多视点的重聚焦特性后，我们将视点质量进一步提升，PSNR均大于30dB；最后，密集视点生成方法在三维光场显示器上完成了实时采集及显示实验验证。基于上述成果，本项目发表SCI论文6篇，专利5项。该项目将在元宇宙等行业拥有广泛前景。