噪声环境下的弱监督图像语义分割研究

Image semantic segmentation is a challenging problem in computer vision. Although fully supervised image semantic segmentation has been shown to achieve promising results, it must make use of pixel-level labels (hard to access in real-world applications) as full supervision. Hence, recent efforts have been made to exploit image-level labels for image semantic segmentation, considering that image-level labels are much easier to access than pixel-level labels. However, when image-level labels provided initially come from social images or are predicted by image classifiers, such weakly supervised image semantic segmentation may be severely affected by the noise in initial image-level labels. In fact, this noise issue has been rarely considered in previous work. In this project, we thus pay our main attention to weakly supervised image semantic segmentation in noisy settings. Concretely, we will first formulate image semantic segmentation as noise reduction over the labels of superpixels based on sparse learning, and then exploit dimension reduction to efficiently solve the resulting large-scale noise reduction problem. Finally, we will predict the labels of test images based on deep learning to obtain weak supervision for image semantic segmentation, when image-level labels are not obtained from social images. The main objective of this project is to achieve some exciting advances in weakly supervised image semantic segmentation and then pave the way for applying them to many challenging tasks in machine learning, patter recognition, and computer vision.

图像语义分割是一个极具挑战性的计算机视觉问题。虽然强监督图像语义分割已经取得大量的研究成果，但是它需要利用像素语义标签作为强监督信息，而这一信息在实际应用中往往难以获取。因此，弱监督图像语义分割开始得到较多的关注，它仅利用相对容易获取的图像语义标签作为弱监督信息。但是，图像语义标签可能来自社交媒体或者由图像分类器预测，所含的噪声会对弱监督图像语义分割产生严重的影响，而很少有工作给出直接的解决方案。因此，本项目拟重点研究噪声环境下的弱监督图像语义分割问题。具体地，我们拟首先利用稀疏学习将图像语义分割建模为超像素的标签去噪问题，进一步利用降维等技术快速求解超像素上定义的大规模去噪问题，最后针对图像标签不来自社交媒体的情形，利用深度学习预测测试图像标签以预先为语义分割提供弱监督信息。本项目有望在基础方法和关键技术方面取得进展，并推动机器学习、模式识别、计算机视觉等相关领域的发展。

图像语义分割是一个极具挑战性的计算机视觉问题。虽然强监督图像语义分割已经取得大量的研究成果，但是它需要利用像素语义标签作为强监督信息，而这一信息在实际应用中往往难以获取。因此，弱监督图像语义分割开始得到较多的关注，它仅利用相对容易获取的图像语义标签作为弱监督信息。但是，图像语义标签可能来自社交媒体或者由图像分类器预测，所含的噪声会对弱监督图像语义分割产生严重的影响，而很少有工作给出直接的解决方案。..本项目重点研究了噪声环境下的弱监督图像语义分割问题。具体地，我们首先利用稀疏学习将图像语义分割建模为超像素的标签去噪问题，进一步利用降维等技术快速求解超像素上定义的大规模去噪问题，最后针对图像标签不来自社交媒体的情形，利用深度学习预测测试图像标签以预先为语义分割提供弱监督信息。此外，本项目对弱监督学习算法进行了推广，成功地将其用于解决遥感图像分类、社交图像标注等实际问题，并且还研究了与弱监督学习紧密相关的两类问题：零样本学习和小样本学习，很好地扩展了本项目的深度和广度。相关成果已经发表了19篇论文，其中在重要国际期刊TPAMI、TIP、TCYB、TGRS、CVIU上发表5篇，在顶级国际会议NeurIPS、CVPR、SIGIR、AAAI、IJCAI上发表7篇。