内容敏感的图像集图像联合分割和联合抠图技术研究

In recent years, image collection segmentation and image co-segmentation have received considerable attention from both the Image Processing and Computer Graphics communities, due to its wide applications to photo editing and advertising design, etc. It serves as a fundamental basis for many multimedia and vision tasks such as compression, depth estimation, annotation, and semantic understanding. Previous techniques such as co-segmentation and iCoseg however often design their algorithms based on some assumption that the images to cut out share similar appearance of foreground and different or similar background. They generally cannot be extend to the segmentation tasks for real photo collections which often contain a certain amount of images with complex and diverse content, and often have translucent foreground objects. This project concentrates on investigating the key techniques and algorithms involved in image collection segmentation, especially co-segmentation and co-matting for the relevant images in a collection with complex image content. We will first develop an algorithm for automatically selecting representative samples from the given image collection. Then, the similarities between the foreground/background of the samples and the underlying foreground/background of target images are to be computed. We furthermore devise a content-aware segmentation algorithm which cuts out the foreground objects from the collection based on the learnt appearance models of the target images. To refine the unsatisfactory segmentation results, we tend to append more user strokes only on a small part of them and co-improve the results locally. The key objective next is to devise an effective co-optimization framework for propagating the refinement information to the rest images with similar incorrect segmentation parts and refining the results locally by taking the updated appearance models accordingly. We will further explore the co-matting techniques for those relevant images with translucent foreground objects. The confidence map for measuring the fidelity of the alpha mat is to be computed by using regression or other machine learning techniques. We plan to formulate the co-matting problem into an optimization framework based on defined objective function. An efficient co-matting algorithm is to be devised. We will develop a prototype which can efficiently produce high-quality segmentation results with few and friendly user interactions and support parallel computation for acceleration. The prototype should be easily generalized as a commercial software. Finally, we will explore the segmentation approaches for segmenting specific image collections such as surveillance and medical images. Based on the achievemtents, we plan to publish 20 papers on well-known international journals and conference, including at least 15 high-quality papers on ACM/IEEE Trans. or top level conferences. We will apply for four invention patents.

图像集分割是近年来图形学与视觉领域的研究热点，针对具有复杂和多样内容图像的图像集的分割仍是一个研究难点。本项目拟围绕图像集中图像的前背景联合分割和联合抠图涉及的关键问题进行探索，具体包括: 研究图像集中具有内容代表性的样本选取方法；探索图像集所包含各图像间关联性计算方法；基于样本的前、背景分割信息及图像间相关性，设计一个行之有效的算法实现图像集中图像的自动联合分割；对于初始分割结果不理想的目标图像，探索切实可行的协同优化方法以改进相关图像的分割结果；提出基于回归分析等机器学习方法的抠图置信度计算模型；定义图像联合抠图的目标优化框架和方程，提出联合抠图方法；研究实现图像集分割的高效并行算法并建立一个原型系统；探索图像集分割在公共安全和医学图像处理等领域的应用。在国内外重要刊物和会议上发表20篇以上论文，包含15篇顶级期刊（IEEE/ACM会刊或其他知名SCI期刊）或会议论文，申请专利4项。

作为本项目研究的核心，图像视频分割是图像视频处理领域经久的研究热点。关于同一前景，人们往往能够获得多张照片或多段视频，不同图像（视频）前景的相似性可以作为前景提取的先验，将其融入分割框架中可以帮助图像视频的前景提取，促进分割问题的解决。我们称针对具有相同或相似前景的多张图像（多段视频）的前景提取为图像视频的前景联合分割。在项目的支持下，课题组围绕图像视频前景联合分割及其应用展开深入研究，取得了系统性进展：.首先提出一种在多张图像间检测相似物体的方法，将该问题转化为特征变换的聚类分析问题，提出了能够针对变换进行聚类的非线性均值漂移算法，发展了均值漂移理论；其次，提出了样本驱动的图像联合分割方法，能对包含一定数量规模图像的图像集实现高效并行分割，突破了前人工作图像背景不同的假设和对所处理图像数量的限制；进而定义了图像协同抠图问题，提出了国际上第一个协同抠图方法，提出了抠图效果的置信度学习模型，采用局部抠图效果互优化的方式协同改进多张相关图像的抠图效果；再次，提出了视频前景的联合分割框架，提出了综合考虑纹理、运动等多维特征的时空超像素子空间聚类方法，是国际上较早能同时提取多段视频共同前景的方法，发布了视频联合分割数据集；此外，在应用方面，将联合分割的思想应用于遥感影像的目标变化检测，将目标物体空间和时间变化特征有机结合并融入联合分割的优化框架，综合利用时空特征检测目标变化。在项目的支持下，课题组还在内容敏感的多图像拼接显示、高光谱图像去噪增强、视频的矢量化以及内容的故事版表示、室内场景三维重建和纹理材质智能推荐等方面取得创新性成果。.报告研究成果的论文发表于IEEE Trans. Image Processing、IEEE Trans. Visualization and Computer Graphics、IEEE Trans. Multimedia、IEEE Trans. Geoscience and Remote Sensing以及AAAI、Pacific Graphics、软件学报等，申请或获授权发明专利多项，取得了较丰硕的科研成果，课题组在图像视频的前景联合分割方面的研究引起了较好的学术反响。课题组较圆满地完成了项目的研究。