基于属性学习的零样本图像分类研究

Because the data distributions between training and testing samples are different, zero-shot image classification becomes a very difficult learning task. In this project, we aim to effectively solve the image classification problem under the zero-shot learning scenario by describing all object classes with common attributes and transferring the attribute knowledge from known classes to new classes. The main contents in our research include the following aspects. A deep attribute learning model based on deep neural networks is constructed to simultaneously realize the deep-level feature representation of images and the training of attribute classifier. The designed deep attribute learning model can not only avoid the subjectivity resulted from manually select features, but also solve the poor generalization problem of attribute classifier due to shallow learning methods. By borrowing the idea of multi-task learning, the attribute classifier (or the attribute ranking functions) and the image classifier are collaboratively learned by using the shared low-level image features. Thus, the attribute prediction (or the relative attribute ranking) accuracy and the image classification accuracy can be simultaneously improved. By using the domain adaptation learning technique, the domain shift problem of attribute learning is solved from different levels (single-source and multi-source) and different perspectives (classifier adaptation and feature representation adaptation). Thus, the obtained attribute classifier that is trained with seen images can accurately predict the attributes of unseen images. The research fruits not only can enrich and develop the existing machine learning theory, but also can be extended to many pattern recognition-related fields.

训练数据和测试数据的分布不同使得零样本图像分类成为一个非常困难的学习任务。本项目拟通过对所有对象类共同的属性描述，将以前学到的各类属性知识迁移到新的对象类上，从而有效解决零样本学习场景下的图像分类问题。具体包括：利用深度神经网络构建一个深度属性学习模型，同时实现图像的深层次特征表达及属性分类器训练，以回避人工选取特征的主观性和解决浅层学习方法导致的属性分类器泛化能力弱的问题；借鉴多任务学习思想，利用共享的图像底层特征来协同学习属性分类器（属性排序函数）和图像分类器，以同时提升属性预测（相对属性的排序）精度和图像分类精度；利用领域适应学习技术，从不同层次（单源、多源）、不同角度（分类器适应、特征表示适应）来解决属性学习的领域偏移问题，以使得在可见类图像上训练得到的属性分类器能够准确预测未见类图像的属性。研究成果不仅可以丰富和发展现有的机器学习理论，而且可以推广应用到模式识别的诸多相关领域。

由于标记样本的缺乏，已标记类别不可能涵盖所有的对象类，这种零样本学习问题场景广泛存在于计算机视觉、图像分类、人脸和语音识别等领。本项目利用深/宽度学习、多任务学习和迁移学习等技术，通过对所有对象类共同的属性描述，将以前学到的各类属性知识迁移到新的对象类上，从而有效解决零样本学习场景下的图像分类问题，主要从下述2个方面开展研究工作：1）深度网络能够从无标签的原始图像中自动提取出具有良好描述能力的图像特征。相较于深度网络，宽度学习系统具有结构简单、易于与其他模型结合等优点。为此，项目组针对轻量型深度网络、新型宽度网络构造等相关问题展开了研究，提出了：基于自注意力机制的生成对抗网络、自适应多尺度图卷积网络、多路径集成卷积网络、权重共享多级多尺度集成卷积网络、基于反卷积特征提取的深度卷积网络、基于监督超图和样本扩充的卷积网络、领域适配CycleGan网络、领域适应宽度网络、基于块对角约束的多阶段卷积宽度网络；2）利用构造的深度和宽度网络，在零样本图像分类方面，提出了：基于深度加权属性预测的零样本学习、基于自适应多核校准的多源域属性适应学习、基于图正则化特征选择的零样本学习、基于多任务扩展属性组的零样本学习、基于多任务混合属性关系与属性固有特征的零样本学习、基于特征原型的零样本学习、基于耦合自编码与高斯混合模型的零样本学习、基于关系有向图正则化的属性三因子分解模型、基于属性核矩阵的生成特征领域自适应模型、基于混合属性的零样本学习、基于加权重构混合属性组的零样本图像分类模型、基于增强属性-特征的宽度属性预测模型。通过研究，项目组取得的成果为：在科学出版社出版专著1部；在国际国内学术期刊上发表/录用论文43篇；授权发明专利7件；培养博士研究生、硕士研究生8名；获江苏省优秀硕士学位论文奖1项。