开集条件下的图像目标分类研究

With the breakthrough of deep learning on computer vision, it has huge improvement on the performance of classifying the image objects in supervised learning with large-scale training instances and closed set. The classification performance is almost as good as humans. In contrast, it is still significant challenging to conduct image object classification for one-shot, zero-shot and open set recognition, as well as identifying the image objects from the vast amount of unsupervised images. The research will study these challenges by semi-supervised vocabulary-informed image semantic embedding algorithm, extreme value learning algorithm, and the discriminative learning in the unified semantic representation space. Particularly, the semi-supervised vocabulary-informed image semantic embedding integrates the image semantic embedding and semantic word vectors into a unified semantic representation space; extreme value learning is utilized to model the probabilistic distribution of the samples of auxiliary classes, and incrementally update the unified semantic representation space and the probabilistic distribution of auxiliary classes. The unseen instances will thus be effectively rejected of being annotated as auxiliary classes. Finally, in the unified semantic representation space, the discriminative learning algorithms will be explored by using the metric learning on Mahalanobis distance, metric distances between probability distributions, and manifold distance in the unified semantic representation space. Finally, this research will enable image object classification in the open set setting. The proposed new methods will be published in the well-known journals and conferences in the field. We will also develop a demo system to promote our research outcomes.

随着深度学习在计算机视觉领域取得突破性进展，有监督、大样本和闭集条件下的图像目标分类性能实现了显著提升，其分类能力接近人的水平。本项目主要研究小样本、零样本及开集条件下图像目标分类这一挑战性问题，对于大量无监督信息的图像也能实现较好的分类性能，具体研究内容包括三个方面：半监督的词汇图像语义嵌入算法、极值学习算法和空间的判别式分类算法。半监督词汇图像语义嵌入算法研究图像语义嵌入与词向量的表示构成统一语义空间；极值学习可对辅助类样本进行概率建模，增量更新统一语义空间，并标记出训练集外类别的测试样本；在统一语义空间，利用马氏距离、概率分布距离、流形距离等度量学习方法研究判别式分类算法，最终实现开集条件下图像目标分类。项目研究成果将发表在知名国际期刊和顶级学术会议上，通过开发演示系统进行算法性能验证。

图像目标分类是自动分析、理解图像内容的关键技术之一，其旨在根据图像内容进行语义标注或图像类别的分类。尤其，近期深度学习在有监督、大样本和闭集条件下的图像目标分类性能实现了显著提升，其分类能力接近人的水平。本项目研究侧重研究了开集条件下的图像目标分类问题，这里的“开集条件”即是指有大规模的图像目标类别的情形，包括没有在训练集出现过的类别。本项目侧重于三点研究内容：（1）设计基于深度神经网络的半监督词汇图像语义嵌入算法，构建统一语义表示空间，设计深度网络端对端语义嵌入学习算法、多视角的语义嵌入、直推式学习算法等，项目发表多篇高质量学术论文；（2）基于深度学习特征的极值学习算法，通过完善深度特征学习框架，研究利用统计学理论（如极值理论等）对训练数据进行概率建模，完善设计深度网络端对端的语义嵌入学习算法，使之可以自动进行增量式的更新辅助类别过程；（3）基于不同距离度量的判别式识别算法，实现并改进开集条件下的判别学习图像识别算法。在图像识别、素描图像识别与检索、行人重识别、人脸识别以及基于图像的三维重建等多个计算机视觉任务的标准数据集上达到或超过学术界目前的识别精度。本项目通过设计了包括基于深度网络嵌入学习的统一语义表示空间、语义嵌入、极值理论在内的研究方案。各项研究内容之间紧密联系，结合在一起，构成一套完整的开集条件下图像目标识别框架。