基于多信息局部相关模型的视频标注研究

Previous video annotation methods are usually limited with requirement for training data, specific annotation types, or global description. Targeting at these problems, this project proposes the multi-domain information based local relevance model for video annotation. Firstly, we conduct research on the domain based clustering algorithm for multi-source internet data, and the cross-domain adaption method for effective recognition. Based on the feature linear transform algorithm, we adapt and employ the multi-domain large data for training for video concepts detection, to yield initial multi-label annotations for the test video. Secondly, the graph based representation is adopted to model the textual and visual information extracted from the test video, based on which the clustering algorithm is researched to obtain the initial semantic local regions of the video and their relevance information. Finally, to integrate the information from online cross-domain data and the test video itself, we propose the local relevance based model, which is semantically more meaningful for multi-label video annotation. Based on the contextual decomposition algorithm, integrating the initial related text and the low level video feature, the test video can be effectively and locally annotated with multi-labels. Through the above methods, exploiting to use more relevance information and more semantically consistent model, this project is expected to provide a new local relevance model based solution to large scale multi-label region-level video annotation.

现有的视频标注方法普遍受限于训练数据要求、特定识别类型或整体性描述，针对于这些问题，本课题开展基于多信息局部相关模型的视频标注研究。具体研究内容：1）研究对多来源网络数据的领域自动聚类，和跨领域数据的适应识别算法，基于特征线性变换的方法，有效利用来源各异的大数据训练语义概念识别器并应用于目标视频，得到视频的多标签初始标注；2）采用关联图表示和结合视频本身提取的文本和视觉多模态信息，基于图聚类获取视频的初始语义性局部区块及其相互关联；3）研究符合语义特点的基于局部相关模型的多标签标注方法，融合网络跨领域数据信息和视频本身信息，基于上下文相关分解学习算法，并结合初始文字标注信息和底层视觉信息，获取全面准确的、局部性的多标签标注结果。本项目通过以上研究对更多相关信息和更符合语义描述的模型的利用，可望为大量有效的视频标注提供基于局部相关模型的新途径。

现有的视频标注方法普遍受限于训练数据要求、特定识别类型或整体性描述，针对于这些问题，本课题开展基于多信息局部相关模型的视频标注研究。具体研究内容：1）研究对多来源网络数据的领域自动聚类，和跨领域数据的适应识别算法，基于特征线性变换的方法，有效利用来源各异的大数据训练语义概念识别器并应用于目标视频，得到视频的多标签初始标注；2）采用关联图表示和结合视频本身提取的文本和视觉多模态信息，基于图聚类获取视频的初始语义性局部区块及其相互关联；3）研究符合语义特点的基于局部相关模型的多标签标注方法，融合网络跨领域数据信息和视频本身信息，基于上下文相关分解学习算法，并结合初始文字标注信息和底层视觉信息，获取全面准确的、局部性的多标签标注结果。本项目通过以上研究对更多相关信息和更符合语义描述的模型的利用，可望为大量有效的视频标注提供基于局部相关模型的新途径。