3D视频智能编码与优化关键技术研究

In recent years, along with the popularization of 3D movies, the development of big data and virtual reality technology, 3D video technology has become a research hotspot in the field of multimedia. Compared with traditional 2D video, 3D video can reflect the depth information of the scene, giving the user an immersive three-dimensional and interactive. At present, the main problem is that the acquisition and transmission of viewpoint depth information is limited by many aspects, and the amount of 3D video data is relatively large. How to achieve accurate and efficient 3D video coding has important theoretical research significance and practical application value. This project focuses on 3D video coding and optimization based on deep learning. From the aspects of depth map generation and enhancement, multi-view video intelligent coding and depth map hybrid coding optimization, the depth map generation enhancement method based on deep neural network and the virtual viewpoint time-space domain joint prediction method based on probability weighted fusion are studied. A depth map hybrid coding optimization method for content characteristics, considering the intra-frame, inter-frame, inter-view, and inter-component correlation of 3D video, constructs an efficient 3D video coding architecture based on deep learning to meet the requirements of 3D system for storage and transmission. The development of this project will effectively promote the development of industries such as 3D display and virtual reality..This project focuses on the in-depth study of three-dimensional video coding and optimization algorithm based on depth learning. Starting from the three aspects of depth map generation and optimization, synthetic viewpoint prediction coding and depth map coding optimization, it explores the scheme of directly estimating accurate depth information from two-dimensional video, and optimizes the depth estimation accuracy through depth learning-based depth estimation optimization technology. Chemicals. Intelligent disparity compensation prediction between viewpoints and joint prediction between space and space are studied. The temporal and spatial correlation of different viewpoints in multi-view video is fully exploited to further improve the performance of multi-view video coding. This paper explores the video characteristics of depth maps and their corresponding texture maps, makes full use of the autonomous learning ability of depth learning, and studies the convolutional neural network classification method combining the characteristics of depth texture video. The development of this project will build an efficient 3D video coding architecture to promote the development of 3D display and virtual reality industries.

近年来，伴随着3D电影的普及、大数据及虚拟现实技术的发展，3D视频技术成为当前多媒体领域的研究热点。与传统2D视频相比，3D视频能够反映场景的深度信息，给用户以身临其境的立体感和交互性。目前主要瓶颈技术在于，视点深度信息的获取与传输受到多方面的限制，同时3D视频数据量相对较大，如何实现准确高效的3D视频编码具有重要的理论研究意义和实际应用价值。本项目重点围绕基于深度学习的3D视频编码与优化进行深入研究，从深度图生成与增强、多视点视频智能编码与深度图混合编码优化三个方面出发，研究基于深度神经网络的深度图生成增强方法、基于概率加权融合的虚拟视点时空域联合预测方法、基于视频内容特性的深度图混合编码优化方法，综合考虑3D视频的帧内、帧间、视点间、分量间相关性，构建基于深度学习的高效3D视频编码架构，满足3D系统对存储和传输的要求。本项目的开展将有效推动3D显示和虚拟现实等领域的技术发展。

本项目主要探索新一代三维视频的低复杂度编码与优化过程中若干未解决的关键科学问题。获得主要成果如下：1）针对视频采集过程中，深度图精度不足影响合成视点质量问题，通过端到端的深度信息估计，构建表面法线、物体边界及稀疏深度图像联合优化方案，提高合成视点质量。2）针对相邻视点冗余信息导致效率过低问题，设计生成模型进行视点间视差补偿预测，建立视点间联合预测编码方法，综合提高预测精度和编码性能。3）针对深度图编码时间复杂度过高问题，构建深度图编码单元分类器，并设计虚拟试点失真估计模型，从而提升深度图编码效率。上述研究成果，有力支撑了三维视频编码系统的开发应用，为探索三维视频编码与优化提供新的理论成果和方法基础。