基于三维信息单元的高质量虚拟视点绘制技术研究

High-quality free viewpoint generation technology is the key to promote free viewpoint video to application. With the development of digital imaging technology, large-scale three-dimensional scene brings huge amounts of data and complex calculations, which presents new challenges to existing virtual viewpoint generation technologies. This project will research on the key techniques of virtual viewpoint generation aiming to getting real-time and realistic virtual viewpoint image.Three-dimensional object space is established, through which the virtual viewpoint generation problem is decomposed to two parts of the construction of 3DIE and the rendering based 3DIE. By calculating the feature of the object space in best scale to build the representation of scale correlation feature for depth image and RGB image. Thus, Feature transfer function from MVD data to 3DIE is established using the constraints.To improve the quality of virtual viewpoint image, Feature Transfer Function from the MVD data to the 3DIE and adaptive local smoothness constraints are used to optimize the objective presented by 3DIE.In the rendering process, the reconstruction kernel based on local feature constraints, combined with parallel processing architecture of the GPU, is developed to enhance the real-time nature of the algorithm. The research results of this project will provide new theories and methods for the free viewpoint video applications and of great significance to the fundamental theories and techniques upgrading in other related fields.

高质量的虚拟视点绘制技术是推动自由视点视频走向应用的关键。随着数字成像技术的发展，大规模三维场景带来的海量数据和复杂计算对现有虚拟视点绘制技术提出新的挑战。本项目以虚拟视点图像生成的真实感和实时性为目标，通过构造由三维信息单元3DIE描述的物体空间，将虚拟视点绘制问题分解为基于区域采样逆映射过程构建物体空间的3DIE表示和基于物体空间的3DIE表示进行绘制两个子问题。通过提取和表示物体空间的尺度相关特征建立约束条件，构建相应视点下从MVD数据到3DIE表示的特征传递函数，以实现参考图像中的几何结构与纹理特征的完整表示与传递，提高虚拟视点图像的质量。在绘制方面，提出基于局部特征约束的重构核函数，结合GPU的并行体系结构进行绘制，增强算法的实时性。本项目的研究不仅为自由视点视频应用中的虚拟视点生成提供新的理论和方法，还为其他相关领域理论技术的研究提供新的研究方法和思路。

随着硬件技术的进步，自由视点视频广泛应用于数字电视、3D电影、视频直播、远程教育、虚拟现实等领域。高效的虚拟视点绘制不仅决定着用户最终的视觉体验，同时也是降低成本、推动自由视点视频走向实际应用的关键。然而，现有的虚拟视点绘制技术所生成的图像质量和效率往往难以满足实际需求。本项目以自由视点视频生成的真实感和实时性之间的矛盾性需求为背景，研究基于深度数据的虚拟视点绘制中的关键问题，对虚拟视点绘制中尺度相关的信息提取与表示、基于三维表示单元的特征传递函数构建以及并行绘制等关键问题开展研究。为解决空洞、伪影等影响视频质量的关键问题提供一系列新技术和新方法，对于推动自由视点视频应用的理论进步和应用发展具有重要意义。