基于多源监测数据融合的云平台故障诊断关键技术研究

Along with the increasing scale and complexity of Cloud platform, the probability of system faults also increases, which has seriously affected the reliability and availability of Cloud platform, lowered the dependability of Cloud platform and restricted the development of Cloud computing. The monitoring data for fault diagnosis are mainly derived from different methods and different points of view. Based on the monitoring data, fault diagnosis for Cloud platform can detect anomalies, diagnose the types of faults and trace the sources of faults in real time. The diagnosis results can facilitate human operators adopting relevant measures and reducing the adverse effects of faults on Cloud platform, thus improving the dependability of Cloud platform..Currently, the researches of fault diagnosis for Cloud platform mostly focus on the methods based on single-source monitoring data. These methods are unable to distinguish the anomalies caused by real faults from the ones caused by no-faults, resulting in high false positive rate and low accuracy of fault diagnosis..Based on the multi-dimensional analysis and fusion of multi-source monitoring data from Cloud platform, this project will pursue researches in several key technologies, including the methods for qualitative analysis and quantitative characterization of system dependability, the topological optimization algorithms for Cloud platform monitoring network, the anomaly detection and localization algorithms for Cloud platform, and the fault diagnosis and fault source tracing strategies for Cloud platform. This project aims at forming a complete system of theory, method and technique for Cloud platform fault diagnosis based on multi-source monitoring data, thus enhancing the real-time performance and accuracy of fault diagnosis. This project will push forward the development and the engineering application of Cloud computing.

随着云平台规模和复杂性的增长，其系统出现故障的概率也随之增加，严重影响了系统可靠性和可用性，降低了系统可信性，制约了云计算发展。云平台故障诊断主要通过获取源于不同方法和不同角度对系统监测的数据，依据这些数据实时检测出异常状态，再诊断出故障类型、故障源等，据此采取相应措施降低故障影响以提高系统可信性。目前的研究主要集中在依据单一来源监测数据的异常检测，其检测结果难以准确区分异常状态是由故障引起还是由非故障引起，依据这样的检测结果进行诊断的误报率较高、准确率较低。本项目依据系统的多源监测数据，在对这些数据进行多维度分析和融合的基础上，深入研究云平台可信性的定性分析和定量描述方法、云平台监测网络拓扑优化算法、云平台异常检测算法和异常定位策略、云平台故障甄别和故障溯源策略等关键技术，形成基于多源监测数据融合的系统故障诊断理论、技术和方法，提高故障诊断的实时性和准确性，推动云计算技术发展和工程应用。

随着云平台规模和复杂性的增长，系统概率出现的故障也随之增加，这严重影响了系统的可靠性和可用性，同时降低了系统可信性，制约了云计算的进一步发展。为了解决常规单一数据源监测结果诊断中存在的误报率高、准确率低、可扩展性差以及无法判断异常起源等问题，我们提出了多源监测数据融合的研究思路，并设计和实现了相关的云平台故障诊断原型系统用于实验和研究。针对云平台自身，本项目提出了分层次的关联关系模型来形式化定义云平台可信性，提出了基于多维度资源平衡的异常检测部署感知检测框架，考虑上下文环境的资源感知的静态拓扑规划算法，以及针对高动态性的动态拓扑调整算法，从云平台出发对整个多源异构异常监测流程提供多源数据的采集、处理以及流程优化。在多源异常监测算法研究方面，提出了基于增量自组织映射（SOM）的云平台异常检测机制，基于在线学习能力的增量局部异常因子算法，基于不平衡样本的SVM异常检测算法，结合独立成分分析（ICA）和贝叶斯分类方法的异常检测框架等。并在随后研究中通过引入拉格朗日优化支持向量机、设定在线学习中的自适应增量处理规则、在数据预处理中加入聚类划分、添加加权的欧式距离等理论和方法，对多源异构数据融合情境下的异常检测算法进行改进和完善。其他的研究成果包括云平台上的多服务资源竞争、实时流数据传输机制、数字轮转算法、大规模数据快速优化选择算法，以及相关算法在疾病管理和预测中的应用。