基于旋转不变Gabor特征的视频动态几何与纹理提取技术研究

Content structure plays an important role in the understanding and processing of videos. Current video characterization and analysis systems rely on image representations based on low-level visual primitives such as color, texture, and motion. While practical and computationally efficient, it is still difficult challenging to bridge the semantic gap between the low-level nature of the primitives and the high-level semantics. Therefore it is interesting to investigate alternative representations and content descriptors based on the texture and geometric structure. We introduce statistical models for two important components of video content, texture and geometric structure, and demonstrate the usefulness of the model with practical applications. First we construct the Gabor feature space, which is important for video pixel similarity computations. We formalize this by using rotation-invariant Gabor filter banks and applying optimization in texture feature space. This Gabor feature space can be further applied to video analysis, we present a simple and efficient video tracking method based on the feature space using rotation-aware Gabor flow optimization. Our approach extends optical flow computation for constructing the Gabor flow to represent the pixel similarity to preserve temporal coherence when applied to video sequences. In particular, our video processing approach is designed in a spatiotemporal manner to keep temporal coherence in resultant video samples. While the highly parallel implementation on GPU is facilitated to achieve realtime performance of tracking the texture and geometric structure along Gabor flow among the video frames, which are also useful for controlling the color refining. We also plan to allow the user to specify the video content using the sketch-based interface on the keyframe images. For each of these techniques, we provide simple closed-form solutions in energy optimization that yield high-quality results. We also suggest ways in which these properties can be used as a basis for intuitive content-based video analysis and processing. While building good models for video is very important, the ultimate measure of success of a given model is the efficiency of the solutions for practical and objective tasks that may be derived from it. And we are sure that this work has a broad application prospects in the industrial, entertainment，military, scientific research，as well as in our daily life．

视频动态几何结构和纹理提取是数字媒体的关键技术之一,广泛应用于影视制作与动画设计。前期研究发现Gabor滤波器通过方向配准，可以实现对于旋转纹理的有效检测。本项目基于Gabor小波与旋转不变特性，探索建立良好纹理结构识别特性的视频域特征空间，并以此Gabor特征对视频中的动态内容进行追踪和分析，从而提高视频编辑与再渲染的质量和处理效率；通过草图设计界面改善用户交互的体验，研究并实现视频特征相关的纹理分析、基于旋转不变的Gabor流的视频几何与纹理信息配准、以及后期增强现实应用与视频渲染的算法系统。本研究将有效提高视频场景的自然纹理的特征分析与提取的准确性、实现基于纹理特征的视频内容的编辑技术、以及保持时间连续性的视频几何与纹理渲染系统。本课题的研究工作旨在通过自然纹理的准确特征选择与提取、通过视频特征空间的构建与高效处理,为数字媒体发展提供新的研究靶点和应用方向。

项目整体实施顺利，各项任务按预定计划完成。项目组高质量的完成了既定目标，共发表高水平论文35篇（其中SCI论文13篇，EI论文11篇）,人次特邀报告，取得了丰硕的研究成果；在课题组建设方面，课题负责人作为获得2013年上海市“浦江人才”奖励计划，作为第二完成人获得2014上海市科技进步二等奖，主要参与人申请国家级工程技术中心，夯实和加强了课题组的研究基础，构建并完善了实验室的研究梯队；在国内外交流方面，课题组成员在ICCV、ACM I3D、SIGGRAPH Asia、CGI、CASA、ICME等知名国内外会议上作论文/短文宣讲，进一步扩大了课题组的学术影响力；在人才培养方面，课题负责人共培养5名博士(在读)，15名硕士(8人毕业)。独立指导与辅助指导2名研究生获得国家奖学金。本项目研究工作已经取得了坚实的成果。在图像的非真实感渲染方面，项目组提出采用非真实感拟合技术，实现了高形象化与结构保持的二微码生成技术，对于不同二维码形象美化算法会有更好的识别率，该研究成果获得Computer Graphics International 2015最佳论文奖。在图像彩色化方面，研究组利用图像的语义分类与理解技术，通过深度神经网络实现对于图像自动彩色化技术，该研究成果发表在ICCV会议中。实现在视频彩色化方面，本项目研究基于Gabor特征的自适应视频彩色化技术，利用自适应KD树进行快速相似度量以及Gabor流的时空匹配算法，在视频空间逐层色彩传播与优化，在确保视频彩色化结果的精确性前提下，降低了时空的消耗，该研究成果在IEEE Trans. CSVT上发表。在图像灰度化方面，项目组提出一种新的基于图像显著度色彩对比与灰度化方法，能够在保持图像色彩对比效果的同时有效实现灰度化降维，相关工作发表在IEEE Trans. Image Processing上；此外在医学图像处理方面，我们提出一种新颖的基于Gabor的眼底图像血管提取技术，能够自动精确地提取眼底血管边沿特征，上述研究工作已发表在期刊Medical Image Analysis。在多视点视频系统开发与应用方面，项目组提出了基于多视点视频的视点实时合成技术，取得较好进展。