视频时空结构深度模型研究及应用

With the explosive growth of video data, it is urgent to utilize intelligence to achieve automatic analysis and understanding of video data. The existing methods are mostly limited to the isolated analysis of video images and therefore incapable of performing complex video analysis tasks in a real environment. The spatio-temporal structured description of video content is very crucial for complex video analysis. It contains a series of structural characteristics such as spatio-temporal association, target, component, and attribute correlations of the video, which form a structural system that spans spatially and temporally from macro to micro of the video. For this reason, this project intends to study deep learning based methods to fully exploit the temporal and spatial structure of complex video and build a spatio-temporal structural deep model of the video. Specifically, it includes, 1) modeling target spatiotemporal structural relationships based on dynamic spatio-temporal graph regional convolutional neural networks and 2) the multi-granularity feature fusion based on the cascading spatio-temporal graph regional convolutional neural networks. On this basis, this project will form a weakly supervised video detection framework based on the deep model with spatio-temporal structures of the video and conduct related applications. It will focus on the challenging issue of complex video object detection, and promote the development and application of deep video object detection methodologies oriented to the spatio-temporal structure.

面对视频数据的爆炸性增长，迫切需要借助智能化手段实现视频数据的自动分析与理解。现有方法大都局限于对视频图像的孤立解析，难以胜任真实环境中的复杂视频分析任务。视频内容的时空结构化描述对复杂视频分析至关重要，它蕴含了视频的时空关联、目标关联、部件关联、属性关联等一系列结构特性，形成跨越时空、从宏观到微观的视频结构化体系。为此，本项目拟利用深度学习充分挖掘复杂视频时空结构，构建视频时空结构深度模型。具体包括基于动态时空图区域卷积神经网络模型的目标时空结构关系建模和基于级联时空图区域卷积神经网络模型的多粒度特征融合。在此基础上，形成基于视频时空结构深度模型的弱监督视频检测方法。本项目将围绕复杂视频检测这个难题展开研究，推动面向时空结构的深度视频分析方法体系形成与应用推广。

本项目围绕复杂视频分析这个难题展开研究，推动面向时空结构的深度视频分析方法体系形成与应用推广。经过一年的研究和探索，形成了包括图像视频的结构化描述和表示、图像特征匹配、多模图像视频特征表示和融合、半/弱监督深度学习算法和应用等一系列的进展和成果，为复杂图像视频内容分析这一难题提供了新的模型与解决思路，达到了项目预期目标。具体地，发表学术论文23篇，申请发明专利5项，培养博士1名，硕士10名，1名硕士的学位论文被评为安徽省计算机学会优秀论文奖。