联合鲁棒信源编码和信道编码的三维视频可靠性传输的研究

Reliable transmission of compressed 3D video is an important research direction in the area of 3D video processing, which has theoretical significance and practical values. Based on the error-propagating behaviours of 3D video, this proposal focuses on the investigation of an error control scheme for 3D video using joint error-resilient source coding and channel coding, aiming at effectively combining the functionality of mitigating error propagation of error-resilient source coding of 3D video and the error correction capability of channel coding. This proposal in-depth investigates the following problem: 1) Based on the characteristics of 3D video coding and warping process, construct an end-to-end view synthesis distortion model; 2) Based on the inherent relationship between error-resilient source coding and channel coding, formulate a joint error-resilient source coding and channel coding framework for 3D video compression and optimization; 3) In the case that the reconstruction quality of texture cannot be scarified too much, develop a joint texture and depth map coding scheme with texture distortion constraint. This research can facilitate the optimal bit rate allocation between source coding and channel coding in both texture and depth components, and thus achieve the balance between compression efficiency and error control, finally improving the picture qualities of virtual view and coded texture in the lossy channel. This research can also bring new insights for digital 3D television and 3D video streaming over Internet, and provide both theoretical and technical support for promoting applied basic research on 3D video coding.

可靠性传输三维视频码流是三维视频处理领域的重要研究方向，具有重要的理论意义和应用价值。本项目针对三维视频的差错扩散特点，研究联合鲁棒信源编码和信道编码的三维视频差错控制方案，有效融合鲁棒信源编码抑制差错扩散功能和信道编码纠正错误的能力。本项目深入研究以下问题：1）针对三维视频编码和虚拟视点绘制的特征，研究端到端的合成失真度模型；2）针对鲁棒性信源编码和信道编码的内在关联性，研究联合信源编码和信道编码的三维视频的压缩与优化框架；3）在纹理图重建质量不能过多牺牲的情况下，研究基于纹理图失真度受限的纹理图和深度图的编码。本项目的研究有助于实现信源和信道码率在纹理图和深度图上的最优分配，达到编码效率和差错控制的平衡，最终改善虚拟视点和纹理图在有损网络传输下的图像质量。本项目的研究成果可为三维数字电视和三维视频流媒体在网络上传输提供新思路，为推动三维视频编码的应用基础研究提供理论与技术支撑。

三维视频是一种新型的多媒体表达格式，如何提高其在网络上的传输鲁棒性是当今亟待解决的问题。针对三维视频编码和差错扩散的特征，本项目深入研究了以下内容：1）基于率失真优化的联合纹理图和深度图的3D视频编码；2）基于可容忍深度图失真度的深度图编码优化方案；3）纹理图失真度受限的三维视频联合信源信道编码方案；4）基于高斯混合模型的三维绘制空洞区域填补方法。大量实验表明，本项目提出的三维视频传输失真模型可以达到相关系数0.85以上的估算精度；提出的深度图编码优化方案可以解决遮挡区域带来的三维视频质量下降的问题；提出的差错控制优化算法可以显著提高虚拟视点的图像质量，并且能够平衡编码视点和虚拟视点的质量。本项目的研究成果可以为具有较高沉浸感的三维视觉系统的基础研究和应用研究提供理论与技术支撑。