面向安防监控的多约束流形学习视频超分辨率技术研究

In recent years, the complex and destabilizing factors in the security landscape are on the rise. Most criminal cases happening in terrible imaging environment leads to fuzzy surveillance videos, which can not meet the need for identification of the criminal investigation. To super-resolve a image or video, traditional methods take only a single source of constraint information, such as the space-time correlation (the reconstruction based methods) or content correlation (the learning-based methods), to regularize the reconstruction process, so that the result is not satisfactory. In order to solve this problem, we propose to combine the space-time constraints and content constraints in the super-resolution process, extending the manifold learning based method from two-dimensional image patch space to three-dimensional video patch space. Besides, the subclass dictionary learning method based on local manifold graph regularization is presented to expand the expressive capability of the three-dimensional video sample space. We also design a high-resolution manifold constrained K-nearest neighbor searching method unaffected by the image degradation process, thus making the reconstructed manifold space and the orignal high-resolution manifold space become much more consistant. To achieve noisy robustness , we propose an iterative neighbor embedding method, in which the reconstruction weight and the neighborhood structure information are updated iteratively. The aim is that the subjective and objective quality of super-resolved video is expected to gain 1.0 dB improvement, which may significantly improve the efficacy of the video surveillance system.

近年来国家安全形势日趋复杂，各类大案要案多发于成像条件恶劣的环境，监控录像模糊，无法满足刑侦业务辨识的需要。现有超分辨率方法仅以时空相关性（基于重建的方法）或内容相关性（基于学习的方法）来约束高分辨率图像的重建过程，约束信息来源单一，重建效果不理想。针对这一问题，提出将现有基于流形学习的方法从二维图像块空间扩展到三维视频块空间，将时空相关性和内容相关性结合起来对超分辨率重建过程进行多约束。在此基础上，提出基于流形图正则局部约束编码的子类字典学习方法拓展三维视频样本空间的表达能力，提出利用不受图像降质影响的原始高分辨率流形结构作为约束，设计新的K近邻选择方法，使重建高分辨率视频片段与原始高分辨率视频片段流形的局部几何结构更加一致，并通过多次重建权重和邻域结构信息的交替迭代，提高算法鲁棒性。研究成果预期将图像重建主客观质量分别提高1个MOS分和1.0 dB，显著提升视频监控系统的使用效能。

近年来国家安全形势日趋复杂，各类大案要案多发于成像条件恶劣的环境，监控录像模糊，无法满足刑侦业务辨识的需要。现有超分辨率方法仅以时空相关性（基于重建的方法）或内容相关性（基于学习的方法）来约束高分辨率图像的重建过程，重建效果不理想。针对这一问题，本项目深入研究了基于柯西正则化约束的超分辨方法；研究了基于原始高分辨率流形结构作为约束的超分辨方法；研究了基于时空相关性和内容相关性多约束的超分辨率方法。具体而言，柯西正则化方法针对稀疏先验过分强调稀疏性而导致高分辨图像合成效果不理想的问题，在目标函数中，对线性重建系数施加柯西约束，同时考虑重建误差项和柯西约束项，使得表示系数呈现较温和的稀疏分布；高分辨率流形结构约束方法利用高分辨率图像的近邻信息指导低分辨率图像查找到较为准确的近邻。该方法同时利用了低分辨率图像流形空间的几何结构信息，以及不受图像降质过程影响的高分辨率图像流形空间的几何结构信息，从而可以更加准确地进行学习与预测高分辨率图像；多约束方法融合二分K均值方法和改进最近特征线方法，分别用于提升构建样本库的速度和扩充样本库并提高样本库的表达能力。我们所提方法取得了初步成果，基本达到项目研究目标的指标要求。项目执行三年以来，我们共发表学术论文6 篇，其中SCI 检索论文3篇，EI 检索论文3篇，申请发明专利6项（含1项授权），培养博士2名，硕士4名，1名博士进入博士后流动站从事研究工作。本项目基本完成了拟定的研究计划。