基于多任务稀疏特征学习的海量图像理解方法研究

As a fundamental problem in the domain of multimedia content analysis, large scale image understanding has drawn the interests of researchers and industry. However, the images on the internet usually have the complex background, high intra-class diversity and big inter-class differences, which bring the great challenge to image understanding. We study the large scale image understanding approach by combining deep learning, sparse coding, multi-task learning and separability analysis. The aim is to automatically and completely annotate and categorize the objects and scenes appearing in the image. The main contents of our research can be summarized as follows. 1) We deeply discuss the key problems of visual polysemia and concept polymorphism in the large scale image understanding, and propose a visual semantic dictionary framework to solve these problems. 2) In order to get more discriminative image representation, we fuse the spatial information into the Restricted Bolzman Machine, and combine with structural sparse coding. 3) To suppress the noise and improve the robustness of model, we propose a multi-task sparse feature learning algorithm with a non-convex constraint to characterize the membership distribution between visual appearances and semantic concepts. 4) Based on the above visual semantic dictionary, the image can be represented into a hierarchical semantic description, and multi-level separability analysis algorithm is introduced to measure the discriminative power of semantic description on the different level.

作为多媒体内容分析领域的一个基本问题，海量图像理解一直吸引着研究人员和工业界的兴趣。但目前互联网图像存在背景复杂、类内散度高和类间差异大等问题给其带来了巨大的挑战。本项目拟结合深度学习、稀疏表示、多特征学习、独立性分析技术对图像理解方法展开研究，实现对图像中出现的物体和场景等进行自动、全面地标注和整理。主要研究内容包括：1)深入讨论海量图像理解中的视觉多义性和概念多态性问题，提出一种视觉语义关联词典的解决思路；2)在图像表示问题上，在受限波尔兹曼机中融合空间信息，并与稀疏表示有机结合，从而得到有强表征能力的图像视觉表示；3)在图像视觉表观与高层语义的关联建模问题上，提出一种鲁棒的多任务稀疏特征学习模型，通过对这种关联关系构建一种非凸约束，来抑制噪声干扰，提高模型的鲁棒性；4)基于提出的视觉语义词典，图像可表示成一种层次化的语义描述，引入独立性分析方法来实现图像不同层面语义描述的距离度量。

网络多媒体数据在互联网时代的信息传播中发挥着越来越重要的作用，其数据总体上呈现出异构性、高噪声、弱标注等特点，这对于其高层语义的分析带来了巨大的挑战。本项目探索了无监督图像特征表示学习、图像表观与语义关联建模、跨模态度量学习等理论方法，研究了深度模型与流形学习的互补融合、多语义间的关联关系、局部结构保持的跨模态迁移机制等关键问题，克服上述挑战，建立多媒体理解框架，为多媒体人工智能计算领域的实际应用提供有力的理论与技术支撑。