面向HMD的高动态范围立体全向视频视觉体验质量评价

Omnidirectional video can provide the user with an immersive experience. High dynamic range (HDR) stereoscopic omnidirectional video can get more details of the scene, and is the direction of the development of multimedia system. The imaging, content representation, display and interaction of the HDR stereoscopic omnidirectional video system are obviously different from the non omnidirectional stereoscopic video systems. The content acquisition of HDR stereoscopic omnidirectional video system will produce HDR reconstruction distortion, splicing distortion, rendering distortion, and so on; the followed processing of projection, coding and tone mapping will lead to more complex multi distortion. All of these result in the decrease of visual experience quality (perceptual quality and visual comfort). Meanwhile, HDR stereoscopic omnidirectional video system usually uses a head mounted display with a near eye display, resulting in more prominent visual discomfort. How to evaluate the quality of visual experience is the core issue in this field. This project will study the problem deeply and establish a visual perception feature model for HMD oriented HDR stereoscopic omnidirectional video. Based on this model, the effects of different distortions/elements on perceptual quality and visual comfort are explored. Aiming at complex multi distortion problem, objective evaluation models of perceptual quality and visual comfort will be proposed. Based on the evaluation models, the optimization method of user side video content reconstruction will be discussed, which provides relevant theoretical methods for solving the problem of HDR stereoscopic omnidirectional video visual experience quality evaluation.

全向视频可为用户获得身临其境的体验，高动态范围（HDR）立体全向视频能获得更多场景细节，是该领域发展方向。HDR立体全向视频系统的成像、内容表示、显示、交互等方式明显区别于非全向立体视频系统。其内容获取会产生HDR重建失真、拼接失真、绘制失真等，经投影、编码、色调映射等处理后形成更复杂的多失真，可导致视觉体验质量(感知质量与视觉舒适度)下降。同时，HDR立体全向视频通常采用近眼显示的头戴式显示器(HMD)，导致其视觉不舒适问题更突出。如何评价其视觉体验质量是该领域的核心问题。本项目将深入研究该问题，建立面向HMD的HDR立体全向视频的视觉感知特性模型，基于该模型探索不同失真/要素对感知质量与视觉舒适度的影响机理，针对复杂的多失真问题，提出有效的感知质量与视觉舒适度客观评价模型；基于评价模型提出用户端视频内容重建的优化方法，为解决HDR立体全向视频的视觉体验质量评价问题提供相关理论方法。

高动态范围立体全向视频系统记录了围绕中心位置的各个方向的场景信息，允许用户自由改变视角，可为用户获得身临其境的体验，是该领域重要发展方向。高动态范围立体全向视频系统包括内容获取、编码、交互显示(包括色调映射与头戴式显示器)等环节，这些环节可能产生相应失真，从而影响用户的视觉体验质量；因此，如何评价该系统视觉体验质量是该领域发展需要解决的核心问题。本项目从分析影响人眼关于高动态范围立体全向视频系统视觉内容感知特性的要素出发，设计了多个面向该系统视觉内容的主观质量/舒适度评价实验、构建了多个主观质量/舒适度评价数据库，为相应的客观视觉体验质量评价模型设计提供了研究依据和性能评价基准；分析了高动态范围立体全向视频/图像的视觉感知特性，探索了高动态范围立体全向视频系统视觉内容的失真机理及其特征描述；结合视觉心理学/生理学理论、数理统计与信号处理方法，从多层面多角度研究和提出了相应的视觉内容客观质量评价模型与视觉舒适度评价模型，探索了基于视觉感知特性的视觉系统内容生成、编码与重建的优化方法；为高动态范围立体全向视频系统的研究与发展，提出了相关理论与方法。本项目在高动态范围立体全向视频系统视觉体验质量评价及其应用方面取得了相应成果，已完成学术论文36篇，成果发表在IEEE Transactions 期刊、Signal Processing、Digital Signal Processing、Sensors等国际权威/重要SCI期刊，以及国内核心期刊和重要国际学术会议(ICIP、ICME、ACM MM 等)上，已申请或授权发明专利12件，培养了博士毕业生/硕士毕业生13名。