基于标签弱标注数据和深度学习的视觉性概念特征自动提取技术研究

Semantic gap has been a long standing research challenge for intelligent image processing problems. In recent years, deep learning has achieved excellent performance on a lot of intelligent image processingproblems, e.g. image classification. However, few work considers using deep neural networksto learn the projective function from low level image features to high level visual conceptual features. Visual concepts are numerous and exhibit complex (hierarchical) relations. Traditional deep learning methods cannot well handle them.Users in Web 2.0 social media Websites have generated a large amount of social tagging data for images. As semantic descriptions of images, tags have a high potential value in guidingthe training of deep models for automatic extraction of visual conceptual features. This project aims to address the automatic extraction of image conceptual features based on weakly labeled tagging data and deep learning. The research goals include: (1) combining clustering and convolutional neural networks to detect and filter spam images; (2) recognizing tags representing visual concepts and removing those which are irrelevant to visual contents; (3) mining (hierarchical) relations among visual concepts and constructing the visual concept hierarchy; (4) studying and developing weakly supervised learning methods for training deep neural networks based on weakly labeled tagging data; (5) studying and developing supervised learning methods for training deep neural networks based on the concept hierarchy, and finally obtaining the projective neural network which projects images into the high level conceptual feature space.

语义鸿沟问题是图片智能处理的主要挑战之一。近年来，深度学习在很多图片智能处理任务上表现优异，如图像分类等。然而，鲜有研究工作考虑利用深度神经网络模型学习从图像低层特征到高层视觉性语义概念特征的映射函数。图像视觉性语义概念繁多且具有复杂(层次)关联性，传统深度学习方法无法很好应对。Web 2.0社交媒体网站蕴含着海量图像标签标注数据，为实现基于深度学习的视觉性概念特征自动提取提供了丰富的潜在指导性信息。本课题将研究基于标签弱标注信息和深度学习的图像语义概念特征自动提取问题。研究内容包括：（1）结合聚类技术和卷积神经网络检测并过滤垃圾图片；（2）过滤非视觉性语义标签；（3）挖掘语义概念的（层次）关联关系，构建视觉语义概念分层结构；（4）研究基于标签弱标注数据的深度神经网络弱监督学习方法；（5）研究基于概念分层结构的深度神经网络有监督学习方法，最终得到从图像低层特征到高层语义的映射神经网络。

语义鸿沟问题是图片智能处理的主要挑战之一。近年来，深度学习在很多图片智能处理任务上表现优异，如图像分类等。然而，鲜有研究工作考虑利用深度神经网络模型学习从图像低层特征到高层视觉性语义概念特征的映射函数。图像视觉性语义概念繁多且具有复杂(层次)关联性，传统深度学习方法无法很好应对。Web 2.0社交媒体网站蕴含着海量图像标签标注数据，为实现基于深度学习的视觉性概念特征自动提取提供了丰富的潜在指导性信息。在此背景下，本项目从以下四方面开展研究工作，取得显著成果：（1）无监督预训练方法：提出了一种新的基于耦合神经网络的无监督区别性特征预学习方法；（2）面向弱标签信息的弱监督深度学习方法：针对不同情况设计相应的弱监督学习方法，形成了一套较为完整的解决方案框架；（3）多视角深度语义特征学习：提出了面向一致性和互补性高层概念特征学习的深度多视角矩阵分解方法。并进一步采用生成对抗思想解决了视角缺失问题；（4）语义空间随时间演化的深度网络模型：提出两种考虑时序变化因素提取多媒体高层特征的神经网络模型，在视频微动作识别和社会媒体推荐领域取得了很好的效果。共发表成果论文23篇，其中中国计算机学会A类期刊/会议论文15篇。培养博士生4人，硕士生8人。