基于上下文感知的互联网社群图像语义理解

Automatic image annotation has emerged as an important research.topic. The aim of our research topic is to annotate semantic keywords automatically.for the social images. Online social media services such as Flickr and Zooomr allow users to share their images with the others for social interaction. In order to understand these social images, the main contributions of the proposed research topic are as follows: (1) In order to make computer vision system have the similar visual perception function, we focus on the image representation based on visual attention model. Specially, the saliency model based on low-rank matrix decompostion will be stuied .(2) We aim to propose a multi-edge graph to model the multiple relationships among the semantic regions of two images. By propagating tag information over the graph structure, we naturally achieve the tag-to-region assignment, leading to more fine tag information while improving the reliability of content-based image retrieval. (3) We investigate how to iteratively and mutually boost image-level and region-level annotation by taking the outputs from one task as the context of the other one. Instead of intuitive feature and context concatenation or post-processing with context, we focus on the context-adaptive classifier which takes the responsibility of dynamically adjusting the classification hyperplane. (4) Tag refinement scheme is studied well that comprehensively explores the interplay of user, data and feature. We further investigate a collaborative image retagging scheme, which propagates each tag over the specific image similarity graph and couples the propagation of different tags through a tag similarity graph. As a pioneering work, this proposal carries out a series of research efforts for processing the social images and these unqualified tags, especially in making use of content analysis techniques to improve the descriptive power of the tags with respect to the image content.

图像语义理解是近年来计算机视觉中一个非常活跃的研究方向。本课题的研究目标是面向互联网社群图像的自动语义理解。主要研究内容:(1)基于视觉注意力机制的图像显著性分析和表示；(2)针对互联网图像级标签的特点，以多边图的图像描述方法为基础，研究图像级标签向区域级标签的传播算法，实现高效的图像区域级标注；(3)挖掘图像语义层间的上下文信息提取与表示，研究自适应的上下文信息融合学习算法，并应用到图像多级语义的协同优化问题中；(4)社群图像包含丰富且存在大量带噪声的标签信息，研究自动图像标签精准化问题。本项目的特色是：(1)基于认知理论的图像表示为图像理解提供有效的表示单元；同时为图像区域级标注提供区域语义重要度信息。（2）以互联网社群图像的视觉特征和标签信息为研究对象，充分挖掘图像语义的上下文信息，提出有效的互联网图像理解框架与模式，并提出新的异构媒体计算方法，为大规模互联网图像提供有效的检索途径。

本课题的研究目标是挖掘图像中上下文语义信息实现图像语义级和区域级层次化标注。课题组成员围绕着既定的研究目标，在低秩矩阵分解、多标记学习方面取得了一系列的理论创新成果。在图像标注、图像分类以及图像相似度度量取得了一系列的应用创新成果。代表性的研究内容包括：1）提出了基于多任务低秩矩阵分解的图像显著性分析算法；2）提出了基于多标记学习的图像标注方法；3）提出了基于层次化稀疏表示模型的多示例半监督学习算法；4）提出了基于视知觉理论的海量社群图像标签自适应排序算法。在预期完成的研究内容基础上，本项目还扩展了基于低秩矩阵分解理论的年龄估计、基于图模型的图像匹配等研究点，为当前大规模图像理解领域中新的热点问题提供了研究基础。. 在本课题的资助下，项目组成员取得的主要研究成果如下：在IEEE Transaction On Image Processing , IEEE Transaction On Neural Network and Learning System，IEEE Transaction On Multimedia等知名国际学术期刊上发表SCI论文10篇；在领域内顶级国际学术会议CVPR, AAAI,ECCV, ACM Multimedia及其他知名国际会议上共发表国际学术会议论文5篇；申请发明专利3项。培养15名研究生，其中已毕业7名。