基于复杂语义的个性化图像集摘要研究

With the rapid development of Internet technology and social media, the amount of digital images on the network increases dramatically. How we can leverage the rich social media information, including different types of objects and relations among these objects, becomes an increasingly important problem. Conventional research on image collection summarization analysis usually focuses on the simple media information, and this makes the technology difficult to deal with users' complex requirements. In this proposal, we target to address the key problems in image collection summarization by leveraging the knowledge mined from image social communities. We propose a method to build a multi-source information fusion by Webpage visual structure analysis and natural language understanding technology. Sample is evaluated by image quality evaluation system. Thus, there is a priori bias when selecting samples. We model the various objects and high-order relations in social media communities by hypergraphs instead of traditional graphs. We propose a new method for representative image selection based on dense sub-hypergraph. Through the user's personal information modeling, we exploit user’s interests and design a scheme for personalized image collection summarization. This proposal will present a personalized system for image collection summarization, which we believe, will advance the development of not only social media mining application but also image analysis and retrieval theories.

随着互联网技术及社会媒体的蓬勃发展，网络图像数据量呈爆炸式增长。我们如何利用丰富的社会媒体资源生成满足用户个性化需求的图像集摘要成为迫切需求。然而，对于现有的基于多模态媒体信息的图像集摘要研究集中于通用的、简单语义概念的研究，这很难满足用户个性化的需求和视觉复杂语义的体现。本课题致力于研究网络图像集个性化摘要的关键技术，通过建立多模态信息体系来实现跨媒体信息的融合，利用层次化媒体数据语义理解方法建立实体-描述-语义的关联。将跨媒体数据进行深度分析并映射到高层复杂语义空间，在语义关联的基础上进行聚合。利用超图对多模态数据及其关系建模，通过密集子图发现模型实现代表图像查找。通过对用户个人信息建模，挖掘用户的兴趣点和偏好，提出满足用户需求的个性化摘要设计方案。本课题拟构建一个个性化图像集摘要生成系统，将有力推动社会媒体挖掘技术和图像分析与检索理论。

随着互联网技术和移动终端的飞速发展，网络用户产生的图像数量呈爆炸式增长。如何有效高效的引导用户快速查询和浏览的图像集合摘要技术成为迫切需求。现有的图像集摘要技术通常集中于简单，对于多模态数据的语义分析上没有进行研究，这使得其难以适应在复杂语义环境下用户的个性化需求。本课题致力于社会媒体复杂语义下的个性化图像集摘要关键技术研究，提出通过语义关联来挖掘社会媒体潜在的网络结构和关系，围绕技术路线中若干有待解决的问题开展研究。.按照项目计划书的规划，我们首先将社会媒体下图像相关数据信息进行采集，抽取社会化媒体信息多模态异构特征，将其表示为四层语义信息，分别是物理概念层、属性描述层、联想语义层和社会情感层。然后分析不同类型的对象和对象之间的关系，建立以对象和关系为核心的多模态信息体系。通过语义关联，充分挖掘社会化媒体潜在的社会网络结构和关系，采用非负矩阵分解随机动态变化网络获得最大似然重构估计，实现多模态信息的语义聚合。利用社交网络图像标签分布的交叠性和一致性特征，形成基于标签传播特性的团结构，利用不同模态的互补性增加复杂语义理解，对多层次多维度的数据建模。最后对用户的偏好信息进行挖掘，实现个性化的图像数据集自动摘要。.项目组成员共发表和已录用论文5篇，申请专利3项。