海量社群图像语义理解关键技术研究

Image semantic understanding has emerged as a hot topic recently. The aim of our research topic is to annotate semantic keywords automatically for the social images. In order to understand these social images, the main contributions of the proposed research topic are as follows: (1) Multi-instance Learning based single image saliency detection algorithm is firstly studied, which incorporate both the bottom-up and top-down strategies to improve the saliency detection performance. In addition, by analyzing the fact the multiple images co-saliency detection can also be formulated as a typical MIL issue, a MIL based co-saliency detection algorithm will be discussed. (2)We aim to propose a simple yet effective multi-label ranking based image annotation algorithm which utilize the tag pairwise ranking information, and a trace-norm regularization is also incorporated to fully investigate the tag semantic correlations.(3) We study how to establish mapping between tags and image regions. By investigate the semantic context information between training and test image regions, we propose a structural sparse representation based region tagging algorithm which simultaneously assign tags to all the regions within a test image with a set of labeled training data. (4) By analyzing the fact that user provided tags are orderless, we aim to propose an all-season tag ranking framework which can handle both the images with and without distinct objects. Saliency detection algorithm will first be utilized to classify the images into attentive and non-attentive categories. Attentive image will be processed by the tag saliency ranking approach emphasizing distinct objects in the image, while non-attentive image will be handled by the sparse reconstruction based neighbor-voting approach.(5) We aim to propose a matrix completion based social image tag completion algorithm, which aims to automatically fill in the missing tags as well as correct noisy tags for given images. We present the image-tag relation by a tag matrix, and search for the optimal tag matrix consistent with both the the visual similarity and pairwise ranking information between observed tags. As a pioneering work, this proposal carries out a series of research efforts for processing the social images and their corresponding social tags, especially in making use of content analysis techniques to improve the descriptive power of the tags with respect to the image content.

本课题的研究目标是面向互联网社群图像的语义理解。主要研究内容：(1)基于机器学习的单幅图像显著性检测和群组图像协同显著性检测算法。(2)研究基于多标记排序的图像级自动标注算法，通过挖掘标签间的配对排序关系和语义相关性，实现一种高效的图像级标注算法。(3) 针对区域级自动标注问题，研究在结构化稀疏表示理论框架下，从输入端和输出端同时挖掘图像区域的语义上下文信息，并据此提升区域级标注的准确性。(4) 针对社群图像中存在的标签序列无序性的特点，通过分析社群图像的显著性特征分布，研究自适应地从标签与图像的语义相关度及标签所对应图像区域显著度角度实现标签排序算法。(5) 针对社群图像标签填充问题，研究在矩阵填充理论框架下，引入矩阵低秩正则约束项和标签配对排序关系，优化得出图像-标签关系矩阵。课题的特色是，以社群图像为研究对象，充分挖掘图像和标签的语义上下文信息，为海量社群图像检索提供有效的检索途径。

本课题的研究目标是面向互联网社群图像的语义理解。主要研究内容：(1)基于机器学习的单幅图像显著性检测和群组图像协同显著性检测算法。(2)研究基于多标记排序的图像级自动标注算法，通过挖掘标签间的配对排序关系和语义相关性，实现一种高效的图像级标注算法。(3) 针对区域级自动标注问题，研究在结构化稀疏表示理论框架下，从输入端和输出端同时挖掘图像区域的语义上下文信息，并据此提升区域级标注的准确性。(4) 针对社群图像中存在的标签序列无序性的特点，通过分析社群图像的显著性特征分布，研究自适应地从标签与图像的语义相关度及标签所对应图像区域显著度角度实现标签排序算法。(5) 针对社群图像标签填充问题，研究在矩阵填充理论框架下，引入矩阵低秩正则约束项和标签配对排序关系，优化得出图像-标签关系矩阵。课题的特色是，以社群图像为研究对象，充分挖掘图像和标签的语义上下文信息，为海量社群图像检索提供有效的检索途径。经过项目组核心成员(冯松鹤、郎丛妍、王涛等人)的研究工作，该项目取得了一系列的研究成果，在包括IEEE Trans. on PAMI, IEEE Trans. on Image Processing, IEEE Trans. on Multimedia, IEEE Trans. on NNLS, Pattern Recognition 等知名国际学术期刊以及ICCV, ECCV, AAAI, ACM MM 等知名国际会议上发表了多篇学术论文，其中国际学术期刊论文发表及录用15篇，国际会议论文发表9 篇，总计24篇。此外，项目组申请了国家发明专利2项，软件著作权6 项。对照面上项目申请书中提出的完成10-15 篇学术论文的指标，该项目超额完成了预期目标。