整合大规模负载的服务器自适应优化节能调度算法

The small improvement of scheduling algorithms in large-scale process may bring large energy-reducing. The energy-efficient scheduling problem of computing resources by considering various factors is a NP-hard problem in general. Although there are a large number of related researches, however considering the uncertainty and dynamic characteristics in the life cycle of deterministic and stochastic workload caused by virtual machines (VMs), there is still a large distance from the optimal results. Physical resources and virtual resources combined with real-time dynamic changing workload raise new and challenging issues, especially inconsistency of resources and user requirements specifications.cause a lot of energy waste, dynamic changing and life-cycle workload restrictions, large-scale computing in Big Data era and related new requirements bring new problems. In the project, we firstly propose optimal algorithms of self-adaptive energy-efficient scheduling by large-scale workload consolidation. Based on existing world-wide research results and the work of our team, we firstly propose VM lifecycle model and divisible capacity configuration technology, to reduce high cost of inefficient current centralized allocation; then use dynamic asynchronous gradual migration to reduce overhead and frequent vibrations caused by a large number of synchronization VM migrations, quantifying and minimizing the number of VM migrations when underloaded or overloaded happen to PMs, creating adaptive distribution and migration functions to automatically adjusts the allocation and migration; Finally with combination of Quality of Service performance requirements, develop open-source self-adaptive energy-efficient scheduling systems, benchmarks and test data sets for the first time.. Taking advantages of both centralized and decentralized scheduling, this project will build more accurate models and optimal algorithms in the theoretical field of large-scale self-adaptive scheduling, and improve energy-efficiency in practice in cloud data centers.

大数据时代大规模负载调度算法的较小质量改进可导致较大能耗节省。综合考虑多种因素的计算资源节能调度问题在一般情况下都是NP-难问题,尤其是在考虑确定性负载和随机负载生命周期特征的研究方面, 业界尚未能很好地解决物理资源与虚拟资源实时动态结合变化带来的新挑战性问题,因而导致大量能耗浪费等突出问题。本项目拟研究适宜大规模负载的离线、在线和随机负载的分布式自适应优化节能算法,提高算法准确度和效率以适应大规模计算高质量高效率需求。首先, 创建生命周期模型和容量可分割配置技术以减少集中式分配的低效率高开销 ;其次,提出动态异步渐进式最小化迁移以减少频繁同步大量迁移造成的开销和网络震荡;最后,综合节能与性能要求研发自适应优化节能调度算法和开源系统、测试基准及数据集。本项目提出结合集中式和分布式调度优点的新思路, 将建立大规模自适应优化调度的理论成果, 并应用于云数据中心实践中节省能耗。

本项目针对整合大规模负载的服务器自适应优化节能调度算法面临的新的挑战，重点研究了适宜大规模负载的离线、在线和随机负载的分布式自适应优化节能算法，构建了生命周期模型和容量可分割配置技术以减少集中式分配的低效率高开销，提出了动态异步渐进式最小化迁移以减少频繁同步大量迁移造成的开销和网络震荡，综合节能与性能要求研发自适应优化节能调度算法和开源系统。具体地，在理论研究方面，项目组提出了提出容量可分割技术、建立离线和在线调度算法理论优化模型和优化值、研究自适应分配选择函数和异步动态迁移函数等技术，在近似比和竞争比上超越了已知的最好结果，提出兼顾性能的节能调度算法, 考虑包括负载均衡、系统响应时间、一段时间内迁移概率和总次数、系统过载概率、能耗总额受限等服务质量需求；在原型系统开发方面，基于项目理论研究成果，开发了整合负载自适应节能调度原型系统，给出上述算法的性能测试基准和仿真数据集。在项目执行期内，项目组在包括JNCA、IEEE IoT、FGCS、CCPE、JOS、Cluster Computing、EAI、 IEEE Service等在内的国际国内重要期刊和会议上共发表和录用高质量论文20余篇(其中JCR1区1篇、2区7篇，申请国家发明专利6项(其中1项已授权)，成功引进1名国家特聘专家，培养了一批在云计算和大数据方向具有创新能力的优秀青年人才。推动了整合大规模负载的服务器自适应优化节能调度以及云计算等相关领域的技术发展，圆满完成了预期目标。此外，项目组还将研究成果成功应用到云数据中心、互联网服务和工业制造等领域，通过促进科研、技术与社会产业融合，取得了显著的社会和经济效益。相关成果还将广泛应用到云数据中心和大数据处理平台等大规模负载资源调度管理应用中，进一步推动我国云计算和大数据相关领域的技术发展，为国家经济建设服务。