兼容噪声标签的弱监督特征学习与图像理解

Deep neural network based image feature learning and efficient expression have become a paramount research topic in the field of computer vision for nearly five years. The rapid development of deep learning has greatly improved the effectiveness of image feature expression. However, the manual annotation of ever larger image datasets is very laborious and time-consuming. In this project, we tend to focus on effective features learning and image understanding from weakly supervised learning framework compatible with noisy annotation data. Specifically, a multi-task deep convolution neural network model is first proposed to both address image-level noisy label cleansing and precise image multi-label classification. Based on the multi-label classification model, we probe into a global average pooling based class activation maps generation scheme for image categories related object region localization. Secondly, we apply saliency maps generated from unsupervised salient object detection models as initial noisy pixel-level annotation, and further design an iterative label purification algorithm to cleanse and produce higher quality salient object detection results based on deep convolution neural network as well as a probability graph model based on conditional random fields. Finally, combined with the research results of class activation map and image saliency detection, we delve into weakly supervised learning based image semantic segmentation to achieve pixel-level image understanding. The research project can probably promote both the development of weakly supervised deep learning and image understanding. Moreover, the research achievement can also be widely used in many application fields of computer vision, such as robot vision, unmanned scene comprehension, etc.

基于深度神经网络的图像特征学习和高效表达已经成为近五年计算机视觉领域的重要研究课题。深度学习当前取得迅猛发展，大幅度改善了图像的特征表达，但需要大规模人工标注样本进行训练，标记数据所耗费的劳动非常巨大。本项目研究兼容噪声标签的弱监督特征学习与图像理解算法。具体研究构建同时支持图像级别噪声标签清洗和图像多标签分类的多任务深度神经网络模型；基于多标签分类模型研究基于全局均值池化操作的图像类别响应映射图生成方法，实现语义类别相关的物体区域定位；将无监督模型生成的图像显著图作为带噪声的像素级别标签，研究基于深度神经网络及概率图模型的迭代式标签优化算法，提高显著概率图的计算精度；最后以图像类别响应图和显著图作为辅助信息，研究弱监督图像语义分割算法，实现像素级别的图像精细化理解。本项目的研究将同时推动弱监督深度学习和图像理解领域的发展，项目成果还可以广泛应用于机器人视觉、无人驾驶的场景理解等诸多领域。

本项目以计算机视觉中的智能图像理解任务为研究背景，重点研究兼容噪声标签的弱监督图像特征表达方法。并基于此，研究包含图像多标签分类、图像物体区域定位、显著物体检测以及图像语义分割等四个图像理解子任务的弱监督深度学习解决方案。本项目的研究思路是构建同时支持图像级别噪声标签清洗和图像多标签分类的多任务深度神经网络模型；基于多标签分类模型研究图像类别响应映射图生成方法，实现语义类别相关的物体区域定位；将无监督模型生成的图像显著图作为带噪声的像素级别标签，研究基于深度神经网络及概率图模型的迭代式标签优化算法，提高显著概率图的计算精度；最后以图像类别响应图和显著图作为辅助信息，研究弱监督图像语义分割算法，实现像素级别的图像精细化理解，并探索将模型学习到的特征推广应用到图像理解的更多领域。课题组按照计划，围绕以上目标开展研究工作，在基于弱监督的显著物体检测 、基于感知区域发现的图像多标签分类及弱监督物体区域定位、基于视觉注意力机制的RGBD物体检测、像素级别图像语义理解以及相关技术在监控场景、人脸检测等具体应用方面均取得突破。相关工作共发表已标注的相关论文39篇，其中IEEE Trans. 期刊论文12篇（含T-PAMI论文4篇），CCF A类会议论文19篇，公开发明专利10项，授权2项。基于本项目的研究成果，项目负责人获得广东省杰出青年基金、吴文俊人工智能优秀青年奖、ACM中国新星提名奖、CCF-腾讯犀牛鸟优秀科研基金以及计算机视觉顶级会议ICCV2019最佳论文提名奖等奖项。