分布式存储系统基于纠删码的可靠访问关键技术研究

The continuing expansion of distributed storage systems makes data failure commonplace. By pre-storing a low degree of data redundancy, erasure coding is increasingly used in current distributed storage systems for fault tolerance. However, the equipment of erasure coding also brings some changes to read/write accesses and failure recovery in storage systems. How to accelerate the failure recovery and mitigate the influence on read/write accesses brought by erasure coding remains an intractable problem. To this end, this project investigates some key issues for erasure-coding based reliable access in distributed storage systems, including 1) we design and realize an optimization algorithm for node recovery in heterogeneous distributed storage systems; 2) we design and realize a data layout for degraded reads optimization and an efficient transition algorithm between different erasure coding schemes; 3) we design and realize an optimization technique for updating erasure-coded data. The main purpose of this project is to speed up the node recovery and improve the read/write performance in erasure-coded distributed storage systems, so as to provide technique support for the application of erasure coding in distributed storage systems.

分布式存储系统规模扩张导致数据失效成为常态。纠删码通过保存少量冗余数据以实现数据容错，被广泛应用于分布式存储系统之中。然而纠删码的引入也改变了系统的读写访问和失效修复方式。如何加快基于纠删码的数据修复并降低其对系统读写访问的影响，成为当前分布式存储系统亟待解决的重要问题。基于此，本项目针对分布式存储系统基于纠删码的可靠访问关键技术展开研究，具体内容包括：1） 设计并实现面向异构分布式存储系统的节点修复优化方法；2）设计并实现面向降级读优化的数据布局和纠删码高效切换算法；3） 设计并实现面向纠删码的数据更新优化方法。本项目主要目标为加快分布式存储系统中基于纠删码的节点修复，并提升其读写访问性能，从而为纠删码在分布式存储系统的应用提供技术支持。

本项目研究了分布式存储系统的数据可靠访问关键技术，加快了纠删码数据修复和读写效率，保证了数据密文检索的高效和安全，并取得如下成果：.（1）设计并实现了跨集群感知的纠删码数据修复和扩展方法，最多能缩短91.5%的数据修复时间和95.2%的扩展时间；.（2）设计并实现了存储系统中的纠删码数据更新方法，至少能够提升40%的更新性能；.（3）设计并实现了基于编码规则的数据布局策略，能够减少32.9%的降级读时间；.（4）设计并实现了面向云存储的密文检索机制，在保证数据安全的同时，促进密文检索更加契合实际应用场景需求。.在项目资助的三年内，本项目组录用和发表国际期刊和会议论文11篇（其中包括CCF-A类论文4篇，CCF-B类论文5篇），其中项目负责人一作论文9篇，培养硕士研究生7名。