基于闪存的软硬件协同设计技术研究

Minimizing data movement is of great significance to improve the performance and reliability of flash-based storage systems. This project investigates the following software-hardware co-design technologies. (1) Research on an ECC-based data deduplication scheme. It exploits the ECC value within flash-based devices to identify duplicated data, thus reducing the redundant data on the write path. This scheme can eliminate the calculating overhead of the MD5/SHA-based fingerprints and the potential hash conflict problem. Therefore, it can improve the performance and reliability of deduplication-based storage system. (2) Research on a reference-count based garbage collection scheme for flash. By considering the reference count characteristics of deduplication-based system when dealing with the garbage collection processes in flash-based storage systems, this scheme places the data chunks with high reference count in the same flash blocks, thus reducing the data movement during garbage collection process. (3) Research on a near-data processing scheme. It exploits the parallelism characteristics and the computing resources within storage devices to offload the query operator into flash-based devices, thus reducing the unnecessary data movement on the read path. (4) Research on applicability of the above key technologies on Non-Volatile Memory (NVM), such as PCRAM, RRAM, MRAM and STT-RAM. Studies of these software-hardware co-design technologies can improve the performance and reliability of flash- and NVM-based storage systems, thus further promoting the applications of these devices in storage systems.

减少数据移动对提高闪存存储系统的性能和可靠性具有重要意义。本项目拟通过软硬件协同设计技术，（1）研究基于ECC的重复数据删除技术，挖掘闪存设备中的ECC校验值识别重复数据，在写路径上减少重复数据写入，消除基于MD5/SHA的指纹计算开销和潜在的哈希冲突问题，提高重删系统的性能和可靠性；（2）研究基于数据引用率的闪存垃圾回收技术，在闪存垃圾回收过程中考虑重删系统中的数据引用率特性，将高引用率数据页布局在相同闪存块中，减少垃圾回收过程中的数据移动；（3）研究基于查询算子卸载的计算存储一体化技术，挖掘闪存介质的并行性特点和设备内的计算资源，将查询算子卸载到闪存设备，在读路径上减少不必要的数据传输；（4）上述关键技术在新型非易失存储器件上的扩展研究。通过上述软硬件协同设计关键技术的研究，提高闪存存储和新型非易失存储系统的性能和可靠性，进一步推动闪存介质和新型非易失存储器件在存储系统中的应用。

项目通过软硬件协同设计技术在闪存和非易失性存储系统的数据写入路径、垃圾回收过程和数据读取过程中减少数据移动量，从而提高了闪存和非易失性内存的性能和可靠性。主要研究了：（1）基于ECC的重复数据删除技术，消除了哈希指纹计算开销；（2）基于数据引用率的闪存垃圾回收技术，提高了闪存垃圾回收效率；（3）基于查询算子卸载的KV SSD技术，提高了数据查询效率；（4）基于闪存和非易失性内存的极致性能优化技术等，并取得了一系列的研究成果，在中国计算机学会CCF推荐会议和期刊上发表论文18篇，其中CCF推荐A类论文9篇，包括HPCA、INFOCOM、IEEE-TC/TPDS/TCAD等，申请技术发明专利7项，已授权3项；相关研究成果获DATE 2019最佳论文提名，核心技术研究方案获2022年CCF-华为胡杨林基金-存储领域专项优秀方案奖和华为公司的价值火花奖。