视频防抖关键性技术研究

Videos captured with portable devices(hand-held cameras, cameras mounted on vehicles) often appear remarkably shaky, resulting in the deterioration of the video quality. The problem becomes serious when the video contains multiple dynamic objects, multiple depth layers and large depth variations. Besides, the rolling shutter effects can further cause content deformations. Traditionally ,these challenges were handled separately, or even stayed untouched, which would cause the artifacts such as wobbling and structure aberration. This project focus on several critical issues of video stabilization. By solving the problems and integrating the solutions, a video stabilization system will be accomplished, which is robust to various types of videos. The research includes, (1) a method of 3D rotational and translational motion separation that deals with the presence of discontinuity depth; (2) a 3D co-reconstruction based scheme that stitches multiple videos for the enlarged view angles; (3) a patch-match, space-time fusion algorithm that enhances the blurry frames; (4) and a feature based approach that rectifies the rolling shutter distortions. We will study how to combine the proposed techniques into a unified framework, from which the high quality videos can be obtained. We will validate the system through extensive quantitative and qualitative evaluations.

使用手持、车载或机载设备拍摄视频不可避免会产生大大小小的抖动，直接导致视频质量下降。这一问题在复杂场景下（多个运动物体、多重背景、大景深等）尤其严重。另外，卷帘快门形变也会导致视频变形。传统的视频防抖方法只能应对单一难点，然而在实际拍摄中，多个难点都是捆绑出现的，应统一应对。本课题拟研究视频防抖中的几个关键技术，从而建立一套具有普适性、鲁棒性和可扩展性的视频防抖系统。研究内容包括：(1) 设计基于三维旋转平移分离的方法来应对非连续深度场景的运动恢复和渲染；(2) 拓展组三维重建技术来完善多视频协同防抖及拼接；(3) 采用块搜索和时空融合来消除视频模糊；(4) 基于特征点匹配的卷帘快门形变矫正。本课题将把上述几个关键技术有机的结合在一起，使得各点之间能相互促进，组合成一套具有实用性的视频防抖系统，并辅之以大量的实验进行验证和测试。

本课题研究了视频防抖中的几个关键技术，建立了一套具有普适性、鲁棒性和可扩展性的视频防抖系统。具体研究内容包括4个方面：(1) 提出了一种网格流算法MeshFlow, 可进行实时在线视频防抖和图像对齐；（2）提出了基于旋转平移分离的近景视频防抖算法。(3) 拓展组三维重建技术实现多视频协同防抖及拼接；(4) 采用块搜索和时空融合技术实现视频去模糊。本课题把上述几个关键技术结合在一起，组合成了一套具有实用性的视频防抖系统。