基于内容感知的RGB-D视频非规则形状缩放方法研究

The videos processed by existing content-aware video resizing technology can only be displayed in the display devices with rectangular regions, which does not meet the diversification requirements of the display devices with non-rectangular regions. Meanwhile, with the commercialization and wide application of depth sensors, the captured RGB-D videos bring big challenges to the video resizing technology. This project will take the RGB-D videos as research objects, and focus on the content-aware video resizing technology, including the important region detection method based on adaptive bounding box, the irregular shape driven video resizing method based on seam carving, and the irregular shape video enhancement method based on scene-space perception. This project will solve the existing problems of content-aware video resizing technology, including the important region inaccuracy detection, resizing shape regularization, and unclear resizing details. The research results will construct efficient and fast RGB-D video processing system or similar systems, and further expand the scope and depth of content-aware video resizing technology, therefore, this project is of great significance and application value.

现有的基于内容感知的视频缩放只能将处理后的视频展示在矩形区域显示设备上，无法满足非矩形区域显示设备的多样化展示需求。同时，深度传感器的商业化和广泛应用，其所拍摄的RGB-D视频对视频缩放技术带来了更高的挑战。本项目以RGB-D视频为研究对象，重点研究基于自适应包围盒的视频重要区域探测、基于裁剪线的非规则形状驱动的视频缩放和基于场景空间感知的非规则形状视频增强等内容感知的视频缩放技术，解决传统内容感知视频缩放所面临的重要区域探测不准确，缩放形状规则化，细节不清晰等问题。本项目的研究成果对构建高效、快速的RGB-D或类似视频处理系统，进一步拓展内容感知视频缩放技术的研究范围和深度，具有非常重要的研究意义和应用价值。

本项目主要研究如何将RGB-D视频或重要区域高质量的显示在不同尺寸和形状的屏幕设备上，包含视频帧的重要区域探测方法、重要区域保持下的非规则形状视频缩放实现及视频缩放结果质量增强三个方面。已取得如下成果：(1)提出基于自适应流形滤波的显著细节探测方法和采用跳层卷积神经网络的显著性检测方法，提高重要区域的探测准确性；(2)提出重要区域约束的非规则形状展示方法和感知驱动的非规则形状视频缩放方法，实现缩放形状的多样性；(3)提出基于时空显著区域的视频增强方法，提升了非规则缩放结果的细节质量；(4)构建了非规则形状视频缩放模型，开发了面向RGB-D视频的非规则形状缩放系统。上述成果实现了RGB-D视频在不同形状设备上的不缺失和不失真的高质量展示，可适应多种场景的商业化需求。