面向在线数据密集型应用的高效数据存储与复杂查询关键技术研究

Online Data-Intensive (OLDI) applications, includes Web search, E-commerce, and social networks, have recently become popular in industrial and academic fields. In addition to the "big data" property of traditional data-intensive applications, OLDI applications have several distinguishing properties such as latency constraints and query condition diversity, which lead to urgent demands on data access lantency performance and data query diversity (e.g., multi-attribute range query, Skyline query). This project proposes to conduct research on several aspects of OLDI applications including data storage, network transmission, and complex queries. 1. According to the low latency requirement, we will study technologies on large-scale network RAM based key-value (KV) store, which uses RAM as a persistent storage to support efficient data access, and realizes high persistence, availability and consistence through Glocalization (= globalization + localization), globalizing data storage/backup and localizing failure detection/recovery. 2. Aiming at the property of latency constraints of data transmission, we will research technologies on deadline-aware application level network virtualization, which supports network isolation and sharing among multiple OLDI applications, and realizes deadline-based traffic scheduling and data transmission within one OLDI application. 3. According to the complex query requirement, we will design load-balanced order-preserving mapping mechanism between multi-dimensional data and storage resources, study KV-based distributed index structures, and realize delay-bounded complex queries.

以网络搜索、电子商务、社交网络等为代表的在线数据密集型（OLDI）应用逐渐成为工业界和学术界的研究热点。除了传统数据密集型应用的"大数据"特点外，OLDI应用还具有延迟受限、查询条件多样等特性，对数据访问的延迟性能和数据查询的多样性（如多属性区间查询、Skyline查询等）提出迫切需求。本项目将针对OLDI应用的存储、网络和查询等几方面开展研究。1．面向数据访问的低延迟需求，研究基于大规模网络内存的键值（KV）存储技术，通过全局化存储/备份以及本地化失效检测/恢复，实现高效的KV数据访问。2．针对数据传输的延迟受限特性，研究最后期限感知的应用层网络虚拟化技术，实现多个OLDI应用之间的网络隔离和共享，以及同一OLDI应用内部基于最后期限的数据流调度和数据传输。3．面向复杂查询需求，设计负载均衡的多维数据到存储资源的维序映射机制，研究基于KV的分布式索引结构，进而实现延迟受限的复杂查询。

以网络搜索、电子商务、社交网络等为代表的在线数据密集型（OLDI）应用逐渐成为工业界和学术界的研究热点。除了传统数据密集型应用的特点外，OLDI应用还具有延迟受限、查询条件多样等特点，对数据访问的性能（如带宽、延迟等）和数据查询的多样性（如多属性区间查询、Skyline查询等）提出迫切需求。.本项目针对OLDI应用的存储、网络和查询等几方面进行了深入研究。首先，面向数据访问的低延迟需求，研究了基于大规模网络内存的键值（KV）存储技术，通过全局化存储/备份以及本地化失效检测/恢复，实现了高效KV数据访问。第二，针对网络拥塞导致数据访问延迟急剧增加的问题，研究了基于无阻塞模型的应用层网络虚拟化技术，实现多个OLDI应用之间的网络隔离，以及同一OLDI应用内部最后期限感知的数据传输。第三，面向复杂查询需求，研究了基于KV的分布式索引结构，设计了负载均衡的多维数据到存储资源的维序映射机制，进而实现了延迟有界的复杂查询。.在本项目的支持下，项目负责人作为第一作者在USENIX NSDI（中国大陆科研机构历史第三篇）、IEEE INFOCOM、IEEE Transactions on Networking、IEEE Transactions on Services Computing等国际会议和期刊发表多篇论文。应邀担任Trans. Services Computing“虚拟化与服务”专刊客座编辑，担任IEEE ICDCS、ICWS程序委员会委员，担任IEEE LSCA、JointCloud主席、CCF优博论坛执委会主席。相关技术已物化进入实际信息系统，在国家的重要业务部门中得到成功应用（应用证明详见http://nicexlab.com/applications_nsfc13.pdf），得到英国皇家学院院士Jon Crowcroft教授高度评价。申请人2013年获军队科技进步三等奖。