社会化视频语义分析关键技术研究

The rapid advances of social network and video capture devices have spawned social video. We have witnessed an explosive growth of social video in recent years. Effective techniques for analyzing social videos are highly desired. However, conventional video analysis techniques cannot well analyze social video data, which contains complicated patterns and diverse content. In this proposal, we target to address the key issues in social video semantic analysis and propose an effective solution. We first propose to construct a high-quality training set, which is used for learning semantic concepts, by leveraging social tags. The media entities, such as images and video clips, are collected through their associated social tags from multiple Web sites. The collected entities are then denoised to form the training set, based on which, we propose to build the models of semantic concepts by simultaneously using multi-source samples and multi-modality features. The models are in turn used to infer the semantic of social videos. We then propose to mine the social context of videos and use social context to help optimize the semantic analysis results. We also propose to incorporate human interaction into social video semantic analysis. An active learning approach is developed to refine the models of semantic concepts. The approach takes into account the properties of social video data. Our research will surely make an important advance in the theories and applications of video analysis and it will play an important role in next-generation video management and service technologies.

社会网络的兴起和视频采集设备的普及催生了社会化视频数据，其规模呈爆炸式增长。高效的社会化视频语义分析技术成为迫切需求。然而传统的视频分析技术无法有效地分析模式复杂、内容多样的社会化视频数据。本课题致力于研究社会化视频语义分析中的关键技术。拟通过社会标签收集多来源网络媒体数据，对数据进行清洗去噪，为语义概念建模构建高质量的样本集；在此基础上，综合利用多源样本以及多模态特征建立高性能的语义概念模型；进而深入挖掘社会化视频的社会上下文信息，合理融合上下文信息对视频的语义分析结果进行优化；并引入人机交互，提出结合社会化视频数据特性的主动学习方法，对语义概念模型进行更新和完善。本课题将有力推动视频分析理论和应用的发展，为新一代视频服务与管理提供核心算法和技术。

按照项目计划书的规划，项目组深入研究了社会化视频语义分析关键技术，取得了较为丰硕的研究成果，实现了预期研究目标。项目组成员共发表学术论文31篇（含录用待刊1篇），其中ACM/IEEE汇刊及CCF A类国际会议长文16篇，申请国家发明专利5项。培养国家优秀青年科学基金获得者1名。部分代表性成果包括多模态视觉特征学习、多模态视觉协同显著性检测、视频内容语义分析、视觉内容描述以及视频相关性分析与推荐等。本项目在理论方法研究成果基础上，设计开放了一套社会化视频语义分析原型验证系统。