视频复杂语义分析关键技术研究

With rapid advances of video capture devices and Internet technology, we have witnessed an explosive growth of video data. Effective video semantic analysis techniques are highly desired. However, conventional research on video analysis usually focuses on the detection of simple concepts, and this makes the technology difficult to deal with users' complex requirements, such as complex-query video search. In this proposal, we introduce a framework for complex-semantics-oriented video analysis. We propose a method to build the models of complex semantic objectives by bridging the "semantic gap" with multiple layers. More specifically, we bridge the semantic gap between low-level features and complex semantic objectives with three types of mapping: (1) the mapping from low-level features to simple semantic concepts; (2) the mapping from simple semantic concepts to concept bundles; and (3) the mapping from concept bundles to concept bundle sequences. The solution involves a lot of novel techniques, including: the analysis and representation of complex semantic objectives; the modeling of a set of simple semantic concepts by aggregating multiple information sources; the modeling of semantic concept bundles and concept bundle sequences; a user interaction approach that supports multiple labeling paradigms and an active learning method that integrates multiple sample selection criteria. Our research will surely make an important advance in video analysis and retrieval, and it will display an important role in the next-generation video management and service technologies.

随着视频数据采集设备的普及和互联网技术的发展，视频数据量呈爆炸式增长。高效的视频语义分析技术成为迫切需求。然而，由于传统的视频分析研究通常集中于简单语义概念检测，这使得其难以处理用户的复杂需求，例如查询复杂的视频检索。本课题致力于研究视频复杂语义分析中的关键技术，提出通过层次化处理"语义鸿沟"来实现复杂语义对象的建模，并围绕技术路线中若干有待解决的问题开展研究。具体内容包括：对复杂语义对象进行分析，抽取简单语义概念并分析它们的时序关系；基于多信息源融合建立简单概念的高性能模型库；基于简单概念的模型，对语义概念群进行建模；对语义概念群序列建立基于概率分布距离的区分式模型；提出结合多种标定模式的人工交互方法以及结合多种样本选择准则的主动学习方法。本课题将有力推动视频分析的理论和应用，为新一代视频服务与管理提供核心算法与技术。

随着视频数据采集设备的普及和互联网技术的发展，视频数据量呈爆炸式增长。高效的视频语义分析技术成为迫切需求。然而，由于传统的视频分析研究通常集中于简单语义概念检测，这使得其难以处理用户的复杂需求，例如查询复杂的视频检索。本项目致力于研究视频复杂语义分析中的关键技术，提出通过层次化处理“语义鸿沟”来实现复杂语义对象的建模，并围绕技术路线中若干有待解决的问题开展研究。.按项目计划书的规划，我们研究了视频复杂语义分析中的若干关键问题。部分代表性成果包括智能训练数据搜集、基于触屏数据的视觉显著度分析、自适应稀疏表示、语义表示空间构建及映射、智能视频拼接等。.项目组成员共发表ACM/IEEE汇刊论文27篇，申请了8项发明专利。培养国家优秀青年科学基金获得者1名。