数据中心网络中时空结合的低时延流量调度机制

With the rapid increase in the number of Internet users and cloud services, data center networks have drawn extensive attention. Currently, most data centers take the Clos topology to connect a large number of servers. In order to fully utilize network redundant bandwidth and reduce the completion time of delay-sensitive flows, data center network traffic scheduling mechanisms have attracted much attention from industry and academia. The existing traffic scheduling mechanisms could be classified into load balancing (selecting flow transmission path) and flow scheduling (determining the flow transmission order). However, due to the data center special traffic characteristics like a large amount of short flows and frequent traffic changing as well as the difficulty in determining the priority of flows, existing traffic scheduling mechanisms face severe challenges. Moreover, the existing traffic scheduling solutions generally consider the two aspects of selecting flow transmission path and determining flow transmission order separately, ignoring their mutual influence. In this project, we first propose a load balancing mechanism at end hosts based on LSTM and PPO machine learning algorithms. Then, we propose a flow-length prediction mechanism based on clustering algorithm, and use the convex optimization theory to set reasonable priority threshold. Finally, we propose a low-overhead distributed spatial and temporal traffic scheduling mechanism. The load balancing and priority-based flow scheduling are jointly optimized to improve network overall throughput and reduce the completion time of delay-sensitive flows.

随着互联网用户数量和云业务的急剧增长，其背后支撑的数据中心网络也得到了广泛关注。当前大部分数据中心采用Clos拓扑结构连接大量服务器,为了充分利用网络冗余路径带宽资源以及减小时延敏感流的完成时间，数据中心网络流量调度机制得到了工业界和学术界广泛关注。现有流量调度机制主要从负载均衡（选择流传输路径）以及流调度（确定流传输顺序）两方面展开。然而，数据中心网络中短流多、突发频繁的流量特性以及流优先级不易确定等特点使得现有流量调度机制面临严峻挑战。而且，现有流量调度方案通常单独考虑流传输路径和流传输顺序这两方面问题，忽略了它们之间的相互影响。本课题首先研究基于LSTM和PPO机器学习算法的源端负载均衡机制；然后提出基于聚类算法的流长预测机制，并利用凸优化理论设置合理优先级门限；最后，提出低开销分布式的时空结合流量调度机制，联合优化负载均衡和优先级流调度，提高吞吐量，减小时延敏感流完成时间。

数据中心是我国算力基础设施，为数字经济发展提供重要支撑。数据中心网络流量调度是影响网络性能的关键技术，为提升数据中心网络冗余路径的带宽利用率并减小业务完成时间，本项目取得的主要研究成果包括：1）针对已有可编程负载均衡器等概率随机分发到后台服务器导致的负载不均问题，提出了一种后台资源感知的可编程负载均衡器，单台P4设备可承载超100万条连接。2）针对现有基于x86的软件负载均衡器时延大，可编程负载均衡器存储不足的挑战，提出了一种基于循环布隆过滤器的可编程负载均衡器，实现状态压缩2.7倍。3）针对流量调度中感知全部路径状态开销过大问题，分析负载均衡成效与探测路径数量的关系模型，并基于模型提出部分路径感知的动态流量调度算法，实现非对称拓扑下流完成时间比CONGA降低33%。在IEEE/ACM Transactions on Networking、IEEE INFOCOM、IEEE ICNP等CCF A/B类或网络领域重要期刊和会议上发表论文20篇（SCI 7篇，EI 13篇），其中第一作者/通讯作者12篇，并获ICCC 2019最佳论文奖。授权专利8项。项目负责人获2019北京邮电大学首批“1551”托举人才、2020教育部霍英东青年教师基金、2021中国电子学会创新团队奖（第5完成人）等荣誉。