跨媒体互联网社群图像语义理解

Social image analysis has emerged as an important research topic. The aim of our research topic is to annotate semantic keywords automatically for the social images. Online social media services such as Flickr and Zooomr allow users to share their images with the others for social interaction. In order to understand these social images, the main contributions of the proposed research topic are as follows: (1)We aim to combine sparse representation with multi-edge graph representation to model the multiple relationships among the semantic regions of two images. By propagating tag information over the graph structure, we naturally achieve the label-to-region assignment, leading to more fine tag information while improving the reliability of content-based image retrieval. (2)By analyzing the fact that user provided tags are often orderless, we aim to propose an all-season tag ranking framework which can handle both the images with and without distinct objects. Visual attention model is adopted to classify the images into attentive and non-attentive images. Attentive image will be processed by the tag saliency ranking approach emphasizing distinct objects in the image, while non-attentive image will be handled by the sparse reconstruction based neighbor-voting approach. (3) The tag refinement scheme is studied well that comprehensively explores the interplay of user, data and feature. We further investigate a collaborative image retagging scheme, which propagates each tag over the specific image similarity graph and couples the propagation of different tags through a tag similarity graph. As a pioneering work, this proposal carries out a series of research efforts for processing the social images and these unqualified tags, especially in making use of content analysis techniques to improve the descriptive power of the tags with respect to the image content.

社群图像语义分析是近年来计算机视觉中一个非常活跃的研究方向。本课题研究面向海量社群图像的自动语义理解。主要研究内容:(1)针对社群图像标注信息在图像级粒度这一特点，将稀疏表示理论与多边图表示机制相结合，研究自动地将图像级标注信息向区域级传播的算法，以得到更加有效的区域级语义标注信息；(2)针对社群图像中用户提交标签的无序性，通过分析图像的显著图特征分布，自适应地从标签与图像的语义相关性和标签对应区域的视觉显著性角度，实现社群图像标签排序；(3)针对社群图像中用户提交标签的语义不精确性，从标签精准化角度重点研究基于视觉和语义信息一致性的标签过滤、标签改善和标签扩充等三个方面。项目拟在视觉认知理论和文本语义计算方法的指导下，以海量社群图像及其文本标签为研究对象，充分挖掘社群图像的视觉特征与标签语义的相关性，提出有效的社群图像理解框架。

随着互联网Web2.0技术的快速发展，出现了大量的为用户提供各种关系、交流的网络社交平台。网络社交平台导致在线社群图像的规模呈爆炸式增长。标签是当前标注网络图像的重要方式。但是，由于受用户知识背景和主观感情的影响，导致标签具有三方面的问题：大多数标签仅仅是在图像级别上对图像内容进行描述，没有考虑每个标签与图像内部区域的语义对应关系；大多数标签是无序的，不能很好地诠释图像的内容；存在标签是模糊的、充满噪音的现象，不能准确的描述图像的内容。针对这些问题，本项目进行了三方面的研究。针对图像区域标注提出了基于多示例学习的区域标注算法、基于稀疏表示的区域级标注算法以及基于前馈上下文和形状先验的区域级标注方法。针对标签的无序问题，提出了基于代价敏感的图像标签自动排序算法。针对标签不完善和存在噪音的问题，本项目利用图像视觉信息和标签语义信息一致性进行标签的改善，包括概念空间上的标签改善和多模态互增强的标签改善。大量的实验验证了所做工作的有效性。本项目已经在重要国际期刊上发表21篇学术论文，申请4项专利和4项软件著作权。研究成果可用于图像检索和自动图像语义分析，具有非常重要的理论意义和广阔的应用前景。