基于深度无监督分簇的混合监督图像语义分割方法研究

Semantic Segmentation is one of the most important techniques in the field of computer vision. Currently, deep learning based semantic segmentation has achieved great progress and it has been widely applied in the fields such as medial images analysis or autonomous driving. General speaking, a large amount of accurate annotations are required for training a segmentation network. However, collecting large-scale accurate pixel-level annotations is time-consuming and typically requires substantial financial investments. Hence reducing human annotation efforts is meaningful for improving the design efficiency of semantic segmentation methods. Semantic segmentation algorithm will confront several challenges such as generating incorrect localization or coarse object boundaries when there doesn't exist enough accurate annotations for the segmentation network training. In this project, we study the solutions for the above issues. Starting from mining the clustering information of images, the project explores the network architectures for unsupervised clustering. Based on studying the relationship between deep unsupervised clustering and segmentation network, novel loss functions will be proposed for weakly supervised segmentation. Furthermore, this project will propose new methods for proxy annotations evaluation and refinement based on common information mining between images. Finally, it will study the hybrid supervised training strategy for semantic segmentation based on the mechanisms of high-quality annotations selection and iterative annotation adjustment. After the implementation of this project, it is not only helpful to achieve breakthroughs in solving the existing issues related to semantic segmentation, but also improves the relevant theories and methods in the field of image semantic segmentation supervised by incomplete annotations.

图像语义分割是机器视觉领域重要的基础算法，现阶段基于深度学习的语义分割已取得重大进展，并被应用于医学影像分析、无人驾驶等领域。语义分割模型的训练需要大量的精确标注样本，但数据的精确标注是耗时和昂贵的工作，减少数据标注的工作量对提高算法的设计效率具有非常重要的意义。但在缺乏精确标注样本的条件下，语义分割模型的训练会面临缺乏准确语义信息和目标级监督信息的挑战。本项目将以对图像分簇信息进行深入挖掘为出发点来设计研究方案，首先将使用网络结构对无监督分簇算法进行建模，以提取更精确的图像分簇信息；然后对无监督分簇与语义分割的关联进行建模，以设计新的弱监督语义分割损失函数；为了对生成标注进行选择和精炼，将以图像共同信息挖掘为基础来设计相应的算法；最后将以高质量样本选择和分割校正机制为基础，提出混合监督式语义分割训练框架。项目研究的目的是为非完全标注条件下的语义分割算法提供新的理论基础和解决方案。

弱监督语义分割是指在仅使用图像标签等弱标注的条件下，进行语义分割模型的训练，从而极大减轻数据标注的成本。但因为缺少精确的像素级标注，弱监督语义分割算法的性能尚不能达到全监督分割算法的性能。为了解决这一难题，本项目从高质量样本选择和原型学习两个角度设计了系列弱监督语义分割算法。主要研究内容包括设计基于深度无监督分簇的图像分割算法，设计基于高质量样本评价与选择的弱监督语义分割算法，设计基于类别原型特征计算的弱监督语义分割算法等。针对以上研究内容，提出了多种弱监督语义分割算法，并已发表4篇SCI论文。所设计的算法，为弱监督语义分割领域的发展提供新思路，如从噪声生成样本中挖掘出更可靠的监督信息已成为主要的技术研究方向。此外所设计的算法已经应用到医学影像分析和钢材表面缺陷检测等领域，体现了本项目的应用前景。