脑认知融合的3D视频质量评价深层模型研究

Research on 3D Video Quality Assessment (VQA) is to predict the human’s Quality of Experience (QoE) on perceived 3D video signal, which serves as the service performance measurement and more importantly the optimization criterion of whole 3D multimedia industries. Therefore, it has profound theoretical significance as well as practical value for real 3D video systems. 3D video quality assessment will study human’s perception on 3D video quality from multiple dimensions: image quality, depth quality, comfort, naturalness and sense of presence. However traditional video quality assessment solutions lack models simulating human brain cognition mechanism, without the ability to reflect the deep level procedure inside human brain on how human beings perceive 3D video signals and form quality perception. In order to break through the traditional framework of simple direct mapping from image features to perceived visual quality, this research aims to construct a deep framework simulating human brain cognition of 3D video perception. The framework will integrate video signal and collected data from brain cognition, and build a 3D VQA model reflecting human brain 3D visual cognition process, which should also be applicable in real 3D video applications. Major topics of this research work include: 1) Building a dataset including all 3D subjective video quality testing results and corresponding brain cognition data features, the scale and property of the dataset need to be designed for practical usage; 2) Research on the theoretical framework of brain cognition integrated deep 3D video quality assessment model; 3) Research on optimization of 3D VQA model in different real 3D video applications. The objective of this research work is to build and develop the theoretical model of brain integrated 3D VQA model, and optimize it in different video applications. The research is expected to contribute substantially to the development of 3D video industry and related standardization work on 3D video quality assessment.

3D视频质量评价的研究能对未来3D多媒体产业提供服务质量评判的标准和优化目标，具有重要理论指导意义和实际科学应用价值。该研究要从多维度衡量人脑对视频质量的感知，但现有视频质量评价方法缺乏拟合人脑认知的模型机制，无法反映脑认知形成视频质量感知的深层过程。为突破传统方法上图像特征到视频质量模型的简单直接映射关系，本课题拟构建模拟人脑认知的深层架构，融合视频数据和反映人脑内部中高层认知的数据，建立符合3D视频应用需求并拟合人脑认知过程的3D视频质量评价模型。本课题拟开展的研究包括：1)建立满足实际应用的3D视频质量主观评价和脑认知数据特征集；2) 研究脑认知融合的3D视频质量评价模型的深层理论架构；3) 研究3D视频质量评价模型在应用中的优化。本课题拟通过上述研究，建立发展脑认知融合的3D视频质量评价深层模型理论，优化其在不同视频领域的应用，促进3D视频产业的发展并推进制定相关视频质量评价标准。

3D视频质量评价的研究能对未来3D多媒体产业提供服务质量评判的标准和优化目标，具有重要理论指导意义和实际科学应用价值。本课题的研究从多维度衡量人脑对视频质量的感知，构建模拟人脑认知的深层架构，建立符合3D视频应用需求并拟合人脑认知过程的3D视频质量评价模型。本课题开展的研究内容包括：1)建立满足实际应用的3D视频质量主观评价和脑认知数据特征集；2) 研究脑认知融合的3D视频质量评价模型的深层理论架构；3) 研究3D视频质量评价模型在应用中的优化。本课题建立了立体视频质量评价数据集并公开给学术界使用。基于脑视皮层立体视觉认知理论，本课题提出了一个针对3D视频的深度感质量评价方法，该方法结合了自回归预测视差熵度量(ARDE)和用于反映视频内容差异的时空域特征能量加权。同时我们据此提出了一个无参考的3D视频整体体验质量评价模型，通过模拟人的双目视觉相加相减操作，在相加通道上使用表征图像纹理失真的融合自然场景统计特征(FNSS)，同时在相减通道上使用视差图的ARDE特征用于反映3D深度感。我们提出的3D视频质量评价模型在公开的三个3D视频数据库上均优于现有的3D视频质量评价方法。本课题还构建了对应于人眼左右视通路机制的双通路多层交互神经网络，融合分层隐变量特征建立3D图像质量感知模型，在多类别失真数据集上与人眼感知度量相关性高达97%。课题也完成3D视频质量评价模型的原型系统，并建立3D传输原型系统，验证本课题模型的有效性。在国内外主流学术期刊(如IEEE Trans. on CSVT, IEEE Trans. on Image Processing等)和主流国际会议(如ICME, ICIP, QoMEX, VCIP, ISCAS等)发表高水平论文28篇；申请国家专利11项，提交标准技术提案4项。参与组织2次国际学术会议（ICIP 2017和PCS 2019），分别担任组织委员会成员和技术委员会(TPC)主席。本课题通过上述研究，建立和发展了融合人脑认知的3D视频质量评价深层模型，优化了其在不同视频领域的应用，为后续视频产业的发展奠定了良好的基础。