语义感知的闪存管理关键技术研究

In the era of big data, data volume and the number of data-intensive applications are rapidly increasing, requiring higher and higher flash storage capacity. As flash storage capacity rapidly increases and multiple file systems share the storage, flash storage systems face more complicated I/O patterns and flash management. Due to the limitation of the standard block device interface, flash storage systems cannot perceive the file systems semantics that impact I/O patterns and thus cannot efficiently optimize flash management based I/O patterns. Hence, the performance, lifetime, and fairness of flash storage systems decrease, which becomes a serious problem. To solve this problem, it could be a good solution by mining and utilizing file systems semantics. In this project, we first analyze the relationships between core designs of file systems and I/O patterns, mine relevant semantics, and design methods that pass semantics to the flash storage system. Then, we analyze the relationships between I/O patterns and efficiency of flash management and between I/O patterns and efficiency of resources allocation in the flash storage system. Based on the analysis, we investigate semantics-aware address translation technique, garbage collection technique, and resources (both storage space and I/O time budget) allocation and I/O scheduling technique. Finally, we build a real prototype of the flash storage system to evaluate the proposed techniques. The achievements of this project will provide new solutions and key technologies on improving the performance, lifetime, and QoS (quality of service) of flash storage in computer systems such as data centers, mobile terminals, and embedded devices.

大数据时代数据量和数据密集型应用快速增长，对闪存系统的容量提出越来越高的需求。随着闪存容量快速增加和多文件系统共享存储资源，闪存系统面临更复杂的I/O负载特征和闪存管理。由于块设备访问接口的局限性，闪存系统难以对文件系统的访问特征进行语义感知并根据访问特征优化闪存管理，造成存储性能、寿命和服务公平性下降，成为一个亟待解决的问题。因此，挖掘和利用语义是一条可能解决上述问题的新途径。本项目首先在文件系统层分析核心机制与I/O负载特征之间的关联关系，挖掘相关语义和建立语义传递机制；然后在闪存系统层分析I/O负载特征与闪存管理效能、资源分配效率之间的关系，研究语义感知的地址转换技术、垃圾回收技术、时空资源分配和I/O调度技术；最后构建真实的闪存系统原型来评估和验证这些技术。本项目研究成果将为提高大容量闪存系统在数据中心、移动终端和嵌入式设备等场景下的性能、寿命和服务质量提供新的解决方案和核心技术。

在大数据存储需求驱动下，闪存固态盘正在替代传统机械硬盘成为主流存储设备。然而，由于传统块设备接口的局限性，上层软件与闪存设备之间存在语义鸿沟，导致存储系统的性能和寿命优化面临瓶颈。针对这个问题，本项目围绕语义感知的闪存管理关键技术，从地址映射、垃圾回收、以及资源分配与I/O调度三个方面展开研究工作。在地址映射方面，提出感知存储负载重复数据语义和数据库日志数据语义，利用闪存地址重映射特性，在保证数据一致性基础上消除闪存上重复数据写入。在垃圾回收方面，提出感知文件系统和哈希索引场景下数据生命周期语义，实现冷热数据分离以减少闪存垃圾回收过程中数据迁移；另外提出在保证可靠性基础上选择性利用闪存copyback高级命令加速垃圾回收过程中数据迁移。在资源分配与I/O调度方面，提出感知多租户语义，通过闪存空间资源分配实现租户间性能隔离和总体性能优化；另外提出感知QLC闪存页读延迟差异特征，负载自适应地动态缓存读数据，以最大化读性能。以上研究成果为提高闪存存储系统的性能和寿命提供了新颖的技术思路和有效的解决方案。