目标外观剧烈变化场景下的视频跟踪和分割研究

Video Tracking tracks the moving object while video segmentation extracts it from each frame. They are the basic and key problems in intelligently analyzing videos and mining their values. Therefore, video tracking and segmentation have great values and widely applications. However, the present methods have difficulties in dealing with the large target appearance change, which finally lead to inaccurate results. This project proposes a new appearance representation model based on the long-term features of target by developing the independent changing model of target components and preserving the structure of target. A new temporal-spatial saliency measure is defined based on the minimum spanning tree and the short-term features of target which are constructed from the adjacent frames. Based on the above appearance representation model and the temporal-spatial saliency measure, this project combines the topology structure, the long-term and short-term features to construct the target cues. Using these cues, a new video tracking method is proposed by defining the bidirectional verification scheme and the layered sampling method. This tracking method can break through the limitation of the one-direction verification scheme and the present sampling method. Additionally, a new video segmentation method is proposed based on the “predict-segment-optimize” segmentation modelm which is defined by the object flow and target cues. The proposed video segmentation method can reduce the errors produced by the present methods which do not predict or optimize the target. Finally, this project deals with the challenges from large target appearance change to achieve accurate video tracking and segmentation by preserving the coherence of target appearance and the consistency of target movement. The research result of this project is in favor of improving the intelligent analysis of videos, and in favor of promoting the intelligent developments in the related fields such as auto-driving, virtual reality, intelligent agriculture and so on.

视频跟踪和分割能够追踪和提取运动目标，是视频数据智能分析和价值挖掘的基础和关键，具有重要研究意义。当前主流方法常难以应对自然场景中目标外观的剧烈变化，容易导致视频跟踪和分割失准。本项目通过研究目标部件的独立变化模式和目标拓扑结构关系的保持，设计基于目标长期特征的表观模型；通过构建刻画目标新状态的短期特征，设计基于最小生成树的时空显著度；并结合以上两者构造融合拓扑结构、长期与短期特征的目标线索，进而设计双向验证和分层采样构建视频跟踪算法，突破当前单向验证和采样方法的局限；同时，根据物体流和目标线索设计基于“预测-分割-优化”模型的视频分割方法，以减少当前因缺乏目标预测和优化环节而产生的分割误差。最终，通过保持目标的外观一致性和运动连贯性应对其剧烈外观变化挑战，实现精准的视频跟踪和分割。研究成果将有利于提升视频智能分析技术，有利于推动自动驾驶、虚拟现实、智能农业等相关领域的智能化发展。

视频跟踪和分割能够追踪和提取运动目标，是视频数据智能分析和价值挖掘的基础和关键，具有重要研究意义。当前主流方法常难以应对目标外观的剧烈变化，容易导致视频跟踪和分割失准。.项目组通过以下4步展开研究。第一步：针对视频跟踪，从如何有效利用目标的长久稳定特征、目标的整体性线索、目标局部的表达能力、交互引导价值等出发，构建了多种视频跟踪方法，提出了：基于多尺度分块学习的跟踪方法、基于多特征融合的跟踪方法、基于懒交互的跟踪方法、基于历史线索和物体性的跟踪方法。这些方法在目标剧烈形变或长久遮挡时，能够准确再识别出物体，提高了跟踪准确度。第二步：针对视频分割，从如何融合中层视觉线索和底层视觉线索出发，构建了基于超像素匹配和缝络传递的视频分割方法，该方法能够有效应对目标外观剧烈变化时分割结果的漂移，减少了分割误差。第三步：针对视频数据的低质问题，基于编解码结构、残差学习、多层迭代卷积、超像素二元次表达等，构建了视频数据的去雨和去雾算法，提升了视频数据质量，为视频分割与跟踪提供了必要的预处理方法。第四步：探索了视频分割和跟踪的应用，提出了生猪个体的实例分割算法和生猪的眼肌分割算法，实现了非接触式生猪的生产状态评估；并基于视频跟踪，提出了牛羊的行为识别和评估。这些应用指导了精准饲养和科学育种，促进了智慧农业发展。.针对以上研究，在视频跟踪和分割方面，已发表相关学术论文14篇（第一作者10篇，通讯作者3篇），申请发明专利7件（授权4件，审查3件），获得软件著作权5项。培养1名副教授获得教授，1名讲师获得副教授，4名研究生和6名本科生毕业。实施国际合作交流3次，指导学生获省级以上奖励3次和创新创业项目3项。项目主持人获中国图象图形学学会科技进步奖1次，广东省农业技术推广奖1次。相关成果已在农业龙头企业温氏食品集团有限公司的10家种猪场应用。