保持全局形状和视觉舒适度的2D和3D媒体适应方法研究

The ubiquity of mobile digital devices and the rapidly increasing availability of stereoscopic 3D devices demand the technologies to optimize the display of digital media on these devices. This project proposes to research and develop 2D and 3D media retargeting technologies that adapt input digital media to heterogeneous devices with different sizes and aspect ratios other than originally intended. This project focuses on key problems in media retargeting, including global structure distortion, temporal incoherence in video, and visual discomfort of stereoscopic 3D media. We propose to preserve the global structures by dividing an input image into a non-uniform adaptive mesh such that each salient feature is enclosed only in one mesh element and warping each salient feature as a whole. We plan to analyze the effects of different camera motions on shape preservation and temporal coherence, and utilize the motion history information to optimize shape preservation and temporal coherence. Since visual comfort is crtical for stereoscopic 3D media, we propose to borrow stereoscopic photography and cinematography rules, analyze the key factors that affect the visual comfort, and build mathematic models to measure and preserve the visual comfort during stereoscopic media retargeting. Meanwhile, we propose a 3D media retargeting framework that enables the reuse of existing 2D media retargeting technologies. Finally, we will develop a 2D and 3D media retargeting infrastructure to integrate, analyze, compare and further advance media retargeting research. Our research will improve the user performance and experience when playing and working with digital mobile devices, make domestic software and hardware industry more competitive in the world, and promote the international and domestic market for 3D TVs, 3D phones, and other emergering 3D devices.

移动数码设备的普遍流行和三维立体设备的飞速发展，迫切需要与之匹配的数字媒体自适应显示软件的研发。本项目以各种2D和3D数字媒体在多种数码显示设备上的高效显示为背景，针对当前媒体适应方法在保持显著特征的全局形状、视频内容的时间连续性和3D媒体的视觉舒适度方面存在的问题展开研究。主要研究内容包括：基于与图像内容相一致的自适应网格的2D图像适应方法；结合运动历史信息和摄像机运动类型、优化形状保持和时间连续性保持的2D视频适应方法；重用和拓展2D媒体适应方法研究中取得的成果、借鉴立体摄像学和摄影学专业知识实现保持视觉舒适度的3D媒体适应方法。并将上述研究成果集成到本项目开发的媒体适应方法原型系统中进行分析、比较和改进。本项目的研究将有助于提高人们使用多种数码设备进行休闲娱乐和工作时的效率和主观感受，增强我国数字媒体行业软硬件的国际竞争力，并扩大3D电视、3D手机等新兴设备的国际和国内市场。

为适应2D和3D手机、显示器、高清电视、电影屏幕等大小各异的显示屏幕，本项目对2D和3D媒体的显示适应方法进行研究。本项目主要工作包括：提出基于与图像内容相一致的自适应网格的2D图像适应方法，该方法能够保持显著特征的全局形状；提出能够避免内容翻转、并优化形状保持和时间连续性保持的2D视频适应方法；提出基于样例学习并增强视觉质量的3D图像裁剪方法；提出基于缝操作并保持视觉质量的3D图像适应方法；提出保持视觉舒适度和3D形状的3D视频适应方法。在完成研究计划的基础上，本项目对图像适应评估方法进行研究，提出一种基于机器学习并与视觉感知一致的2D图像适应结果评估方法。鉴于显著性检测结果对媒体适应结果具有重要的影响，本项目还对显著性检测进行研究，包括建立一个图像显著性检测数据集、提出图像显著性检测方法、提出图像显著性检测优化方法、并提出图像显著性检测评估方法。上述研究成果有助于提高我国数字媒体行业软硬件水平，增强我国数字产品的国际竞争力。