面向Web图像检索的在线多模态哈希技术研究

Web image retrieval has the characteristics of multi-modal, large-scale and dynamic. Current multi-modal retrieval and hashing technology can solve the retrieval of large-scale images, but two important problems still exist. At first, current multi-modal hashing methods cannot use online learning for the dynamic images. The other is that with the change of web images, their hash codes should also be changed. Current online hashing methods only update hash function online, they cannot update the old hash codes of Web images effectively. In this program, we study an online multi-modal semantic hashing method to solve these problems. Fistly, we propose a unified framework which integrates multi-modal correlation, multi-graph learning and hash codes updating, thus the framework can support multi-modal hashing of Web images. Then we study the online learning of hash function and hash codes. Since the computing of hash codes is relevant to the size of database, we transfer the hash codes into a small dynamic matrix for further optimization. At last we will evaluate the performance of our methods on publicly available datasets, as well as our own collected dataset. And we will compare our method with the state-of-the-art hashing methods.

Web图像有着多模态性、海量性与动态性等主要特性。虽然现有的多模态检索以及哈希技术能够有效解决海量Web图像的检索。但依然有两个难点无法得到解决。第一点是当前的多模态检索技术缺乏在线学习的功能以应对动态性，而在线哈希方法也无法处理多模态的Web图像。另外随着Web图像的变化其哈希码也应改变，现有的在线哈希方法仅动态更新哈希函数，而无法动态更新Web图像旧有的哈希码。针对Web图像的特性以及现有哈希方法的不足，本项目研究一种在线多模态哈希方法。首先提出一个统一框架，将多模态关联、多图学习以及哈希码更新三个模块有机整合在一起，从而能够支持多模态Web图像的哈希检索。其次研究哈希函数与哈希码的在线学习。由于哈希码的计算效率受数据库大小影响，因此将哈希码转化为一个较小的动态矩阵进行优化。最后在公开的Web图像数据集和本项目收集建立的数据集中验证项目方案的有效性，并与国际上最先进的哈希算法进行对比。

Web图像有着多模态性、海量性与动态性等主要特性。现有的图像检索方法包括跨模态哈希与在线哈希均无法有效的应对这些特性。本项目研究一种面向多模态Web图像的统一检索框架。该框架包含三个模块：基于半监督离散多任务学习的语义分析，基于深度学习的跨模态语义关联学习，和动态多模态在线哈希。该检索框架有效解决动态增加的海量多模态Web图像的检索问题。本项目以NUS-WIDE和MIR Flickr为基础整理数据，并通过实验验证了检索方法在精确度和时间效率上的效果。本项目提出的检索框架不仅可以应用于Web图像的检索，未来还可以进一步应用于移动平台例如无人机的视频监控任务。