基于多源信息调制孪生神经网络的目标跟踪算法研究

The challenges, such as the limited amount of online samples, sample noises, highly accurate localization and real-time requirements, bring great difficulties to deep learning based object tracking algorithms. To address these problems, we propose a multi-cue steered Siamese networks based object tracking research project. Based on deep learning and recurrent learning, the project will focus on the work in the following aspects via Siamese networks: 1) In terms of the limited amount of online samples, we firstly construct a pre-trained and fast multi-task Siamese network. Instead of updating the pre-rained Siamese features, we utilize a target object and it’s temporal-spatial contexts to rapidly construct a method that can adaptively adjust the pre-trained Siamese features to achieve accurate and real-time tracking. 2) To alleviate the sensitivity of deep learning based models to sample noises, we propose a method for analyzing the quality of online samples using Peak-to-Sidelobe Ratio, fusion of multi-tracker, and object proposal based methods, respectively. As a result, the proposed method can provide supervision to detect occlusion and tracking failures, and thus reduce the samples noise. 3) In terms of improving the localization accuracy of the deep learning based models, we propose recurrent network using temporal-spatial contexts, and explore hierarchical feature fusion methods. The goal of the project is to achieve robust and effective object tracking. The project has high academic and applicable value.

跟踪中在线样本数量的缺乏和噪声、高精度的定位和实时性需求等，给基于深度学习的跟踪算法带来了极大挑战。针对这些问题，我们提出基于多源信息调制孪生神经网络的目标跟踪算法研究项目。该项目在深度学习和递归学习理论的基础上，以孪生神经网络为出发点，拟从以下三个方面开展工作：1）针对样本数量的缺乏问题，拟在离线训练的快速多任务孪生网络的基础上，不更新预训练的孪生特征；而是根据目标和其时空上下文信息，快速学习出能够自适应调整预训练特征的方法，从而保证准确率和实时性。2）在缓解深度学习模型对样本噪声的敏感性方面，拟提出基于峰值旁瓣比分析、多跟踪器融合、以及似物性采样的验证方法，实现对遮挡和表观变化的有效判断，从而降低样本的噪声。3）在增强深度学习模型定位精度方面，拟利用递归网络对目标和其时空信息进行建模；并探索多层次特征融合，提高模型的定位能力。实现鲁棒有效的跟踪。研究成果具有较高的学术和应用价值。

视觉目标跟踪算法在智能视频监控和人机交互等领域有着重要的应用。本课题针对目标跟踪算法中，深度学习模型的大数据依赖性问题、深度学习模型对样本噪声的敏感性问题、深度学习模型的定位精度问题、深度学习模型的实时性问题，开展研究。具体的，经过三年期间，本课题从如下几个方面开展了工作：（1）首先在孪生神经网络框架下，探索如何有效的结合深度学习、回归方法、集成学习、以及融合多层次网络特征来构建鲁棒的基于孪生神经网络的目标表观模型。（2）然后，采用峰值旁瓣比分析、多跟踪器融合、基于似物性采样验证的思路，来实现对遮挡和目标表观变化的有效判断，从而缓解样本噪声对深度学习模型的影响。（3）进一步基于递归神经网络，充分利用视频序列中的时空上下文信息，提高目标跟踪算法的鲁棒性。提高了目标跟踪系统的鲁棒性、可靠性和实时性，具有重要的理论和应用价值。