基于深度学习的跨摄像机长时实时目标跟踪鲁棒性研究

With the development of machine vision technology, the research of long-term target tracking across cameras has attracted more and more attention. In order to improve the long-term effectiveness , computational rapidity and robustness of cross-camera tracking in wide-field complex scenes, and to meet the tracking requirements of complex scenes (such as brightness change, bad weather, dynamic background, camera jitter, etc.), a long-time real-time cross-camera tracking system based on depth learning is proposed in this paper. Firstly, a new collaboration mechanism based on "off-line pre-training + on-line fine-tuning" is established to improve the modeling ability and computing speed of model features. Secondly, the tracker based on motion information matching and the classifier based on appearance information are designed, and the collaborative design and optimization of the tracker and classifier are carried out. Thirdly, an offline pre-training model of template selection based on robust decision-making is established to deal with the method of selecting robust template in the case of "offline". Finally, the shallow features and deep features are fused and an adaptive selection model is constructed. This research will provide academic reference for computer vision workers in the field of long-term real-time target tracking across cameras, enrich the fast deep neural network technology for target tracking, and provide support for further improving robust tracking in different complex scenes.

随着机器视觉技术的发展，跨摄像机长时目标跟踪研究越发受到关注和重视。为了提高跨摄像机跟踪在广视域复杂场景下的长效性（长时跟踪）、计算快速性（实时跟踪）和鲁棒性（鲁棒跟踪），为了满足复杂场景（如亮度变化、恶劣天气、动态背景、相机抖动等）跟踪要求，本课题拟构建基于深度学习的跨摄像机长时实时跟踪系统。首先，建立基于“离线预训练+在线微调”新协作机制以提升模型特征的建模能力和计算速度。其次，设计基于运动信息匹配的跟踪器和基于外观信息的分类器，并对跟踪器和分类器的协同设计与优化。再次，建立基于鲁棒决策的模板选择离线预训练模型以应对“离线”情形下选择鲁棒模板的方法。最后，融合浅层特征与深层特征并构建自适应选择模型。本项目的研究将为计算机视觉工作者在跨摄像机长时实时目标跟踪研究领域提供学术参考，丰富快速深层神经网络的目标跟踪技术方案，为进一步提高不同复杂场景下的鲁棒跟踪提供支持。

跨摄像机长时目标跟踪果的好坏很大程度受限于亮度变化、恶劣天气、夜晚低照度、光照不均匀、动态背景、相机抖动等不确定性因素，因此本课题研究深度不确定性下的智能跨摄像机长时实时目标跟踪算法。首先，本项目构建了基于动态静态特征交叉注意力融合模型的跟踪器设计，该交叉融合模型是一种采用并行策略的软注意力融合算法，可以平行地对双重特征图采用注意力机制并产生最终的融合特征。其次，构建运动信息匹配的跟踪器和基于外观信息的分类器的协作模型，本课题提出基于运动信息匹配的跟踪器和基于外观信息的分类器组群的“离线预训练+在线微调”协同机制。最后，建立基于鲁棒决策的模板选择的离线预训练模型，基于鲁棒决策的模板选择的离线预训练模型设计。本项目的研究成果对于推动长时鲁棒目标跟踪及其相关应用的发展具有重要意义。截止2022年底，已发表高水平SCI论文2篇；申请发明专利3项和实用新型4项；培养在读硕士研究生6名。