基于长短时样本记忆的视觉跟踪方法研究

It is a fundamental problem how an appearance model is learned from the memory of samples. The existing methods for tracking indicate that the performances of the memory of samples in the integrity and reliability also vary with the time span of the samples. Furthermore, it has influence on the adaptability of the appearance model. According to it, the project tries to explore the method for the construction of samples, and do research in the method for visual tracking based on the memory of samples. (1)The long-term memory of feature points is built up. The relationship between the similarities among the samples and the neighborhood in time is analyzed. The model that fuses the temporal correlation into the spatial correlation is proposed in the form of the affinity matrix. It turns into the complete descriptions of the appearance and the pattern of the variation. As a result, the method for tracking based on the match of feature points is established. (2)The strategy of densely sampling short-term samples is analyzed. Aimed at the noisy samples resulting from the circulant shift of an image, a spatial weight function that can be changed dynamically is created in order to suppress the response to the noisy samples. Based on it, both the position correlation filter and the scale correlation filter are constructed for object tracking. (3)The mechanism of the complementation of long short-term memory of samples is studied. The match of feature points and the snapshots of correlation filters are combined so as to design a method for the detection of the change in the appearance. The foundation for the decision of tracking results is derived. This project provides a novel method and a thread for the construction of the memory of samples, thereby pushing forward to solve the problem of modeling appearance.

样本记忆上的外观模型学习是视觉跟踪中的基础问题，现有的跟踪方法表明随着样本时间跨度的改变，样本记忆在完备性和可靠性上的表现也不同，进而影响外观模型的自适应性。据此，本项目拟探索样本构建方法，研究样本记忆上的视觉跟踪方法。(1)根据特征点建立长时样本记忆，分析样本相似性与时序邻近性之间的约束关系，提出关联矩阵形式的样本时空相关性融合模型，形成外观表征及变化模式的完备描述，由此构建特征点匹配的跟踪方法；(2)分析相关性滤波器的短时样本密集采样策略，针对由图像循环平移所引起的噪声样本，设置可动态调整的空间权值函数抑制相关性滤波器的噪声响应，基于此构建位置及尺度相关性滤波器用于目标的跟踪；(3)研究长短时样本记忆互补机制，结合特征点匹配及相关性滤波器快照设计目标外观变化检测方法，导出跟踪结果的决策依据。该项目为视觉跟踪中样本记忆的构建提供了新的方法和思路，对外观建模问题的解决有进一步的推动意义。

视觉跟踪在无人机导航、智能视频监控、工业产品视觉质检等领域中有着较为广泛的应用。而跟踪的性能在很大程度上依赖于跟踪器模型，目前跟踪器模型主要通过样本学习构建。本项目针对跟踪器学习过程中的噪声样本、样本时序关系表征不足问题展开研究, 试图为样本集建立长短时样本记忆。具体研究内容包括：(1)视频特征时序相关性建模方法研究；(2)最大分类间隔相关滤波器的视觉跟踪方法研究；(3) 深度卷积神经网络在视觉跟踪中的应用研究。围绕相关研究主题，项目研究团队开展了一系列研究活动。首先，提出了一个基于视频帧时序相关性的视频单元建模方法，以此解决视频分割问题；随后，提出了一种基于最大分类间隔的相关滤波器，并通过引入样本裁剪达到减少噪声样本、提高相关滤波器判别性的目的；为了提高目标外观特征的判别性，进一步探索了深度卷积神经网络在特征提取方面的有效性；另外，在缓解边际效应问题方面，提出一种施加于相关滤波器的动态空间正则方法，以此抑制相关滤波器对于噪声的响应。最后，在视频数据硬件计算资源的调度方面，提出了一种改进的粒子群算法实现计算资源的负载均衡，为提高视觉任务处理效率提供了方法。另外，探索了平面、3维空间中点云的边界Hough检测方法。本项目为时序样本集的构建提供了思路，对于计算机视频处理有着重要的理论研究意义和工程应用价值，相关技术在工业领域中也有着较为广阔的应用前景。