EB级集群存储性能优化及节点间高效通信方法研究

EB-class cluster storage has become an important direction of cloud storage service. As storage size reaches the EB level, it becomes crucial issue that how to design an efficient management mechanism on the network link, network topology and other infrastructure while the number of storage node, switches and other devices increase.As Software Defined Network (SDN) made a lot of achievements in improving the performance of the network service, this project focuses on some aspects about the methods of performance optimization and reliability of the massive cluster storage service Fusing the SDN technology, they are as follows: first, the approach of the performance optimization and the strategy of the storage node selection and the multipath transmission parallel are studied based on the SDN and a model of multiple attribute decision, comprehensive consideration of factor such as capacity storage nodes and network loads; second, the strategy of the storage node selection for the EB-class cluster storage system based on SDN is conducted, involved the balanced deployment scheme of the controllers of the large SDN network in the EB-class cluster storage environment; third, the monitoring method of the running state of the storage nodes and fault-tolerant strategy in the EB-class cluster storage system with a coordination mechanism of the SDN controllers in the massive cluster storage system are also carried out to enhance the storage service reliability. These key approaches studied in this project make every effort to optimize the performance of the network service, throughput of the network and to increase the availability, reliability and expansibility, which play a role in promoting the development of the cloud storage and have the important theoretical significance and practical application value.

EB级大规模集群存储是云存储重要发展方向。当存储规模到达到EB级时，系统实现不仅仅是对存储节点、交换机节点等设备的简单增加，对网络链路和网络拓扑等基础设施的高效管理机制成为了其中的关键问题。软件定义网络（SDN）技术能大幅提升网络服务性能和质量，本项目拟融合SDN对EB级集群存储服务的性能优化与节点间高效通信方法进行研究，包括：综合考虑存储节点容量和网络状态等决策因素的影响，研究基于SDN和多属性决策模型的性能优化方法和主次存储节点选择、节点间多径并行传输选择优化策略；研究面向EB级集群存储系统中大型SDN网络多控制器均衡部署方案；基于SDN的集群存储节点运行状态监控、以及在SDN多控制器协同机制下的EB级集群存储系统故障处理策略和可靠性方法。本项目所研究的这些方法和技术，力求优化系统中通信网络性能、提升系统可用性、可靠性和可扩展性，对云存储发展起到推动作用，具有重要理论和实际应用价值。

随着互联网业务的高速增长，存储规模不断扩大。当存储规模到达到EB级时，需要将成千上万个节点集成在一起，如何设计网络拓扑、节点间通信方法，以实现对大规模基础设施的高效使用和管理，成为需要解决的关键问题。为此，本项目融合SDN技术对EB级集群存储服务的性能优化与节点间高效通信方法进行了研究：（1）提出了一种融合SDN技术与软硬件协同的大规模集群存储架构。通过SDN技术测量网络状态，支撑存储节点选择和多径传输的优化策略的实现；利用FPGA并行计算的优势，加速控制层面的信息共享和决策，快速响应网络和存储节点故障。（2）提出了基于SDN和多属性决策模型的存储节点选择和节点间多径并行传输的优化方法。考虑包括存储节点容量、CPU利用率、IOPS和节点网络负载等多个决策因素，建立和求解多属性决策模型，优化存储节点和数据转发路径的选择。（3）针对单一控制器难以管理大量SDN交换机的问题，提出了基于FPGA软硬件协同的多控制器部署方案，利用FPGA加速多控制器之间的拓扑信息同步、负载均衡。（4）针对EB级存储系统中网络及存储节点故障多发的问题，设计了基于FPGA的多控制器协作的大规模集群存储系统的容错处理机制，快速响应来自数据平面、控制平面、存储节点的故障。本项目在研究期间共发表论文20篇，其中SCI收录论文10篇，EI收录论文8篇，申请/已授权发明专利共计15件，取得软件著作权2项，并获得广西技术发明二等奖1次。