基于多元图像信息的物体精细识别研究

The accurate representation and invariable abstraction of objects play a crucial role in object recognition due to determining the final performance. Deep Neural Networks (DNN) can approach such a goal to a certain degree through hierarchical abstraction. However, the current models cannot depict yet the physical scenes in real world as accurately as human vision does, and consequently the generalization capability is rather limited for fine object recognition. In this project, inspired by the information processing procedure and working mechanism of human vision, we first propose to take the multivariate images as visual input, including multispectral images, depth image and color images, such that the rich information as captured by human eyes is provided. Then under the framework of deep learning we conduct the theoretical analysis and technique development for fine object recognition. Our goal is to reveal the underlying mechanisms of accurately representing and abstracting objects in images, and design the corresponding DNN models with learning algorithms. .To be specific, we first study how to learn the explicit object shapes, as the shape property is commonly considered to be the most important clue for the object recognition function of human visual cognition. Particularly, it is desired that the hierarchical shape representation can be obtained and the constant abstraction of shapes can be approximately achieved. Second, we investigate how to utilize the visual spatial attention information to generate more accurate object descriptions so that the performance of object recognition can be significantly improved. Concretely, we will explore the production of reliable attention maps, enhancement of attention region processing, and combination of attention information with recognition model. Third, we develop the task-driven learning methods for accurate and rich image features, including the systematical programming of recognition tasks, effective learning of complementary features, etc. Finally, we apply the proposed methods to the sematic analysis of fruit images via two specific visual tasks, i.e., categorization of fruit varieties and evaluation of relative freshness. .In summary, we would build a brain-inspired theoretical framework for fine object recognition and develop a suite of corresponding technical methods when the aforementioned research contents are completed. It is believed that our final achievements would provide a novel and effective solution to realistic applications of object recognition.

物体的精确表征和不变性抽象是决定物体识别性能的关键，深度神经网络（DNN）通过分层抽象能够在一定程度上达到该目标，但当前模型仍无法像人类视觉那样对物理世界进行精确表征，其精细识别的泛化能力依然有限。本项目在人类视觉机制的启发下，融合多光谱、深度、颜色等多元图像信息，研究深度学习框架下的物体精细识别理论和方法，目标是揭示物体精确表征与抽象的底层机制，并设计相应的DNN模型及学习算法。具体地，研究在物体识别中起重要作用的形状属性的学习，实现分层的形状表征及其不变性抽象；研究融合视觉空间注意的物体识别，实现对注意信息的可靠获取、处理增强及识别融合；研究任务驱动的精确特征学习，构建有效的任务规划和互补特征学习模型；将上述方法应用于水果图像分析中，实现水果识别及其相对新鲜度评估。通过这些内容的研究，本项目实现一套基于脑启发的物体精细识别的理论框架及其技术方法，为实际应用需求提供一种新的有效解决方案。

围绕精细化图像识别领域，从学习方法、识别模型、场景应用等三个层次开展了相关研究。在学习方法方面，针对深度学习依赖于大规模精确标注数据问题，研究了不完美数据提供下的深度神经网络学习方法。提出了一种基于跨集合擦除补全数据的小样本学习方法，提高了特征提取器的迁移性；提出了融合联合对抗学习和权重迁移网络的语义分割域适应学习方法，实现了更好的跨域特征对齐；提出了融合精炼和回顾的多任务增量学习方法以及基于新旧数据再平衡的多类别增量学习方法，能够有效保留旧知识的同时学习新知识。在识别模型方面，探索了图像与视频的深度神经网络模型构建技术。提出了一种用于互补特征学习的对偶网络结构，实现了图像细节信息的有效捕获；提出了一种面向卷积层的通道dropout正则化方法，能够有效缓解网络学习的过拟合问题；提出了一种基于局部矫正的视频语义分割方法，能够保持较低计算量下大幅提高分割精度；提出了一种动作边界逐级细化的视频动作检测单阶段网络，实现了动作边界的精确回归。在场景应用方面，除了蔬菜水果数据集外，重点研究了如何利用交通场景先验知识来处理高度动态复杂的交通图像数据。提出了一种基于多项式曲线的车道线建模方法，提出了一种基于语义分割和位置注意力机制的车牌识别方法，提出了一种基于分级回归的压缩域视频车辆计数方法，通过这些创新方法，可有效提升智能交通场景应用的智能水平。.截止目前，发表学术论文23篇，其中期刊论文8篇（含IEEE Transactions论文3篇），学术会议论文15篇（含CCF A类会议论文8篇以及视觉领域顶会ECCV论文3篇）。申请中国发明专利9项，授权中国发明专利1项，获国际学术会议ICPR20最佳论文奖。培养博士毕业生3名，硕士毕业生9名，所指导学生论文获中国科学技术大学优秀博士论文。本项目研究成果具有良好的成果转化潜力，尤其在智能交通图像分析领域，如车道线检测、车牌识别、压线检测等。