视觉目标跟踪中的深度学习表观建模方法

The visual object tracking is a hot topic in computer vision community. Designing an effective appearance model remains a challenging task due to appearance variations caused by illumination changes, object deformation, and partial occlusions. Most recently, deep learning based visual tracking has garnered considerable interest for their great effectiveness in massive tracking data. However, many state-of- the-art methods only focus on the motion and appearance information of the object itself rather than the useful contextual information such as interactive object and scene, which restricts the performance improvement and wide applications of the deep tracker in real scenarios. With the aim of inferring "what, how and where are we tracking", this proposal presents a novel framework for deep appearance modeling. Under this framework, we will explore convolutional layers in different levels characterizing the target from different perspectives. Then we will focus on integrating multiple contextual visual information (i.e., object, background and contextual scene). Regarding these auxiliary objects as the context of the target, the collaborative tracking of these auxiliary objects leads to efficient computation as well as strong verification. In addition, we will study the recurrent neural network to predict the object trajectory and optimize the deep network for the demand of real-time performance. This proposal is significantly important for presenting new theories and methodologies on the deep tracking and also beneficial for developing more technologies and methods for wide applications of visual tracking, e.g., intelligent video surveillance and intelligent human-computer interaction.

视觉目标跟踪是计算机视觉领域的重要研究方向。由于受光照变化、非刚体形变、遮挡等因素的影响，设计一种鲁棒的表观模型一直是视觉目标跟踪的难点。得益于大规模跟踪数据集的建设，基于深度学习的视觉目标跟踪算法在性能上取得了巨大突破。目前的方法主要关注目标本身的表观建模，对目标、背景、时空约束等信息的联合建模考虑较少，且网络规模很难达到实时性的要求。本项目以获取 “什么是目标，目标如何运动，目标出现在哪里”为目的，研究融合上下文信息的深度学习表观建模问题，包括探讨深度网络中不同层级特征的关联关系；研究递归神经网络进行轨迹预测和优化深度网络结构实现实时目标跟踪等。本项目的研究成果将会对深度学习在视觉目标跟踪中的理论发展以及目标跟踪在智能监控、智能人机交互等领域的应用具有重要的促进作用。

得益于大规模跟踪数据集的建设，基于深度学习的视觉目标跟踪算法在性能上取得了巨大突破。目前的方法主要关注目标本身的表观建模，对目标、背景、时空约束等信息的联合建模考虑较少，且网络规模很难达到实时性的要求。本项目以获取 “什么是目标，目标如何运动，目标出现在哪里”为目的，研究融合上下文信息的深度学习表观建模问题。本项目针对视觉目标跟踪中的目标表示、表观建模及更新展开研究，研究成果总结如下：（1）从深度特征的性质出发，本项目从欧式空间到黎曼空间提出了多种目标表示方法，包括多流形的对称正定矩阵距离度量学习、可分解的双线性编码、视频数据的解耦表达等；（2）在表观建模方面，提出了多跟踪器选择策略、深度字典学习表观建模、混合局部和全局的数据关联的多目标跟踪方法；（3）根据目标跟踪结果，本项目进而研究了特定目标的行为理解，提出了一种用于视觉定位的视觉语义图匹配方法；提出了一种场景感知的上下文推理方法以无监督的方式检测视频异常事件；面向图像物体实例搜索的无监督深度量化方法。