多层多域网络化大数据的高效传输理论与方法

The human society is now entering the Big Data Era. Storing, processing and transmitting the huge amount of data in and between data centers pose unprecedented challenges on the existing network infrastructure. The cost of typical data center networks is dominated by the inter-data center network. The current public Internet, designed for fine granular, inter-active but not data intensive applications, is not capable of handling big data transmission effectively. Optical circuit switching, on the other hand, is prone to request blocking and inefficient bandwidth utilization, and hence does not scale in satisfying the omnipresent big data transmission requests in wide area networks. In this project, we show that the traffic in the big data era has two outstanding characteristics, bulk in size but tolerant for delivery delay. Based on these two characteristics, we conduct theoretical and practical research in the control and data planes of future networking paradigms. We propose a switching plane that contains both a high capacity switching matrix, which is capable of elastic bandwidth light-path switching, and a fine granular, small capacity packet switching matrix, so that both bulk and inter-active applications can be served effectively. We further introduce assistive storage into the network nodes, so that the network can take advantage of the delay tolerance of served requests to make full use of switching and link capacity. Detailed subject of research will include: 1) Networking paradigms that best fit the characteristics of big data; 2) Exploring new ways to define network capacity in presence of bulk and delay tolerant data flows; 3) Aggregation and scheduling of big data flows; 4) Protocols in big data transmission networks; 5) Network testbed and migration strategies into the new networking paradigm. The project seeks to publish 25 papers in world’s leading academic journals and conferences, and to file 10 patents. The project will also submit standardization proposals to standard definition organizations (SDOs), such as the Internet Engineering Task Force (IETF).

大数据时代海量数据的存储、处理和传输，对网络基础设施提出了前所未有的高要求。目前基于分组交换或者电路交换的传统网络体制无法提供高效的大数据传输，是大数据应用快速发展的瓶颈。本项目以大数据背景下数据流的两个新特征：数据巨块(Bulk)和延时不敏感(Delay Tolerant)为切入点，围绕“存储、调度和交换(SSS: Store, Schedule and Switch)相结合”的新型网络体制，研究具有大数据特征的传输交换体制、传输性能度量和网络容量理论、跨层数据流汇聚和调度机理、多层多域大数据传输的协议与资源管理、面向新型传输交换体制的演进机制及实验平台等五项内容；重点解决节点存储的网络带宽增益效应、存储调度交换新机制的网络容量理论两个关键科学问题；通过研究实现“四个一”：即“一项标准、一套算法、一组协议、一个平台”；发表高质量论文25篇，申请专利10项，培养研究生25名。

大数据、移动互联网和5G的快速发展，使得网络流量呈爆炸式增长。提高网络效用，降低网络能耗，提高网络的扩展性，成为放在网络领域研究人员面前的重要挑战。大数据时代数据流呈现出两个鲜明的特点：1）少数数据量大的流贡献了绝大部分网络流量；2）数据量大的网络流往往具有延时容忍特性。本项目主要针对这两个特点，对下一代网络的理论和方法进行了系统的研究，重点研究了：1）支持电分组交换/光电路交换共存的混合交换体制；2）支持存储转发的大数据传输网络体制。本项目的主要成果包括：1）提出了一种混合交换网络通用的资源分配框架，把资源分配、性能约束和流量划分等几个元素有机融合在同一个系统中。这种框架提供了研究和运营混合交换系统的全新视角，对于未来此类系统的实用化提供了重要的理论支撑。以此为基础，项目组已经向相关国际标准化组织提交标准草案一项。2）提出了可应用于存储转发大数据网络的框架性路由工具：时移多层图。通过将网络不同时刻的状态表征为一系列快照，时移多层图管理网络时空两个维度的变化规律。大块数据到达网络时，只需要利用时移多层图进行一次最短路径寻路，该数据块在空间上的路径和在网络中不同位置的存储安排，可以一次性地确定下来。时移多层图提供了一个研究和运营存储转发大数据网络的重要工具，对于此类网络的设计、规划和控制协议的制定，提供了重要的理论依据。..本项目发表论文47篇，申请国家发明专利10项，取得软件著作权1项，提交互联网标准化草案1项，培养博士生9人，硕士生23人。项目进一步开发了“交大健康”大数据平台，经过多年的运营，采集包括膳食、运动、健康等维度的数据超过1亿条。