移动闪存的重复数据删除技术

doi:10.19665/j.issn1001-2400.2020.01.018

西安电子科技大学学报 ›› 2020, Vol. 47 ›› Issue (1): 128-134.doi: 10.19665/j.issn1001-2400.2020.01.018

• • 上一篇

移动闪存的重复数据删除技术

贺秦禄¹,边根庆¹(),邵必林²,张维琪¹

^1. 西安建筑科技大学信息与控制工程学院, 陕西西安 710055
^2. 西安建筑科技大学管理学院, 陕西西安 710055

收稿日期:2019-08-20 出版日期:2020-02-20 发布日期:2020-03-19
作者简介:贺秦禄(1985—),男,讲师,E-mail:luluhe8848@hotmail.com
基金资助:
国家自然科学基金(61872284);国家自然科学基金(61672416);陕西省教育厅专项科研(Z20180353);陕西省自然科学基础研究基金(Z20180156);西安建筑科技大学人才基金(RC1707);西安建筑科技大学青年基金(QN1726);西安建筑科技大学自然科学专项(ZR18050)

Deduplication technology for the mobile flash

HE Qinlu¹,BIAN Genqing¹(),SHAO Bilin²,ZHANG Weiqi¹

^1. School of Information and Control Engineering, Xi’an Univ. of Architecture and Technology, Xi’an 710055, China
^2. School of Management, Xi’an Univ. of Architecture and Technology, Xi’an 710055, China

Received:2019-08-20 Online:2020-02-20 Published:2020-03-19

摘要/Abstract

摘要：

由于闪存在智能手机和物联网等设备上的容量和可靠性都是有限的, 重复数据删除可以通过删除重复I/O来解决这些限制,但必须在设备的各种资源限制下完成。提出了一种M-Dedupe重复数据删除方法,应用内容感知聚类重复数据删除关键路径上的I/O请求,通过提高闪存垃圾回收效率来提高手机的性能和效率。通过原型系统验证的结果表明,M-Dedupe在所有选定的移动应用负载下,写数据量平均减少23.7%~42.5%,平均写响应时间减少21.2%~39.8%,平均擦除数据块减少16.8%~43.9%。该方法能够在移动闪存系统中实现高准确率的重复数据删除,提升重复数据删除的效率,节省存储空间。

关键词: 移动存储, 重复数据删除, 闪存

Abstract:

The performance, capacity and reliability of the flash memory on devices such as smartphones and the Internet of Things are limited. Deduplication can resolve these restrictions by removing the duplicate I/O, but must be done under various resource constraints on the device. This paper proposes the M-Dedupe deduplication, which applies content-aware deduplication I/O requests on critical paths, and improves the performance and efficiency of mobile phones by improving the efficiency of flash garbage collection. The prototype system verification results show that the M-Dedupe reduces the write data by an average of 23.7%~42.5%, the average write response time by 21.2%~39.8%, and the average erased block by 16.8%~43.9%. Besides, it can achieve high-deciding duplication in the mobile flash system, improve the deduplication efficiency, and save the storage space.

Key words: mobile storage, deduplication, flash

中图分类号:

TP393.1

贺秦禄,边根庆,邵必林,张维琪. 移动闪存的重复数据删除技术[J]. 西安电子科技大学学报, 2020, 47(1): 128-134.

HE Qinlu,BIAN Genqing,SHAO Bilin,ZHANG Weiqi. Deduplication technology for the mobile flash[J]. Journal of Xidian University, 2020, 47(1): 128-134.

图/表 10

图1

图2

图3

表1

表2

图4

图5

图6

图7

图8

参考文献 23

[1]	CHEN R, WANG Y, HU J , et al. VFlash:Virtualized Flash for Optimizing the I/O Performance in Mobile Devices[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2017,36(7):1203-1214.
[2]	LEE C, SIM D, HWANG J Y , et al. F2FS: a New File System for Flash Storage[C]//Proceedings of the 2015 13th USENIX Conference on File and Storage Technologies. Berkeley: USENIX Association, 2015: 273-286.
[3]	HAHN S S, LEE S, JI C , et al. Improving File System Performance of Mobile Storage Systems Using a Decoupled Defragmenter[C]//Proceedings of the 2017 USENIX Annual Technical Conference. Berkeley: USENIX Association, 2017: 759-771.
[4]	CHEN R, WANG Y, HU J , et al. Image-content-aware I/O Optimization for Mobile Virtualization[J]. ACM Transactions on Embedded Computing Systems, 2016,16(1):12.
[5]	WU S, CHEN X, MAO B . Exploiting the Data Redundancy Locality to Improve the Performance of Deduplication-based Storage Systems[C]//Proceedings of the 2016 International Conference on Parallel and Distributed Systems. Washington: IEEE Computer Society, 2016: 527-534.
[6]	UCHIGAITO H, MIURA S, NITO T . Efficient Data-allocation Scheme for Eliminating Garbage Collection during Analysis of Big Graphs Stored in NAND Flash Memory[J]. IEEE Transactions on Computers, 2018,67(5):646-657.
[7]	KIM Y, IMANI M, PATIL S , et al. CAUSE: Critical Application Usage-aware Memory System Using Non-volatile Memory for Mobile Devices[C]//Proceedings of the 2015 IEEE/ACM International Conference on Computer-Aided Design. Piscataway: IEEE, 2016: 690-696.
[8]	JIN Y T, AHN S, LEE S . Performance Analysis of NVMe SSD-based All-flash Array Systems[C]//Proceedings of the 2018 IEEE International Symposium on Performance Analysis of Systems and Software. Piscataway: IEEE, 2018: 12-21.
[9]	CAI Y, GHOSE S, HARATSCH E F , et al. Error Characterization, Mitigation, and Recovery in Flash-memory-based Solid-state Drives[J]. Proceedings of the IEEE, 2017,105(9):1666-1704.
[10]	MAO B, WU S Z, JIANG H , et al. Content-aware Trace Collection and I/O Deduplication for Smartphones[C]//Proceedings of the 33rd International Conference on Massive Storage Systems and Technology. Piscataway: IEEE, 2017: 514-522.
[11]	CHANG L P, SUNG P H, CHEN P T , et al. Eager Synching: A Selective Logging Strategy for Fast fsync on Flash-based Android Devices[J]. ACM Transactions on Embedded Computing Systems, 2016,16(2):34.
[12]	PARK J, LEE S, KIM J . DAC: Dedup-assisted Compression Scheme for Improving Lifetime of NAND Storage Systems[C]//Proceedings of the 2017 Design, Automation and Test in Europe. Piscataway: IEEE, 2017: 1249-1252.
[13]	KHAN A, LEE C G, HAMANDAWANA P , et al. A Robust Fault-tolerant and Scalable Cluster-wide Deduplication for Shared-nothing Storage Systems[C]//Proceedings of the 2018 26th IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems. Piscataway: IEEE, 2018: 87-93.
[14]	JI C, CHANG L P, SHI L , et al. Lightweight Data Compression for Mobile Flash Storage[J]. ACM Transactions on Embedded Computing Systems, 2017,16(5s):183.
[15]	REN J, LIANG M C J, WU Y , et al. Memory-centric Data Storage for Mobile Systems[C]//Proceedings of the 2015 USENIX Annual Technical Conference. Berkeley: USENIX Association, 2015: 599-611.
[16]	Adler-32算法 [EB/OL]. [2019-03-05].
[17]	WIDODO R N S, LIM H, ATIQUZZAMAN M . SDM: Smart Deduplication for Mobile Cloud Storage[J]. Future Generation Computer Systems, 2017,70:64-73.
[18]	CHEN R, WANG Y, HU J , et al. Image-content-aware I/O Optimization for Mobile Virtualization[J]. ACM Transactions on Embedded Computing Systems, 2016,16(1):12.
[19]	TAVAKKOL A, SADROSADATI M, GHOSE S , et al. FLIN: Enabling Fairness and Enhancing Performance in Modern NVMe Solid State Drives[C]//Proceedings of the 2018 International Symposium on Computer Architecture. Piscataway: IEEE, 2018: 397-410.
[20]	WU F, LU Z, ZHOU Y , et al. OSPADA: One-shot Programming Aware Data Allocation Policy to Improve 3D NAND Flash Read Performance[C]//Proceedings of the 2018 IEEE 36th International Conference on Computer Design. Piscataway: IEEE, 2018: 51-58.
[21]	TAVAKKOL A, GóMEZ-LUNA J, SADROSADATI M , et al. MQSim: a Framework for Enabling Realistic Studies of Modern Multi-queue SSD Devices[C]//Proceedings of the 2018 16th USENIX Conference on File and Storage Technologies. Berkeley: USENIX Association, 2018: 49-66.
[22]	YAN S, LI H, HAO M , et al. Tiny-tail Flash: Near-perfect Elimination of Garbage Collection Tail Latencies in NAND SSDs[C]//Proceedings of the 2017 15th USENIX Conference on File and Storage Technologies. Berkeley: USENIX Association, 2017: 15-28.
[23]	NGUYEN D T, ZHOU G, XING G , et al. Reducing Smartphone Application Delay through Read/Write Isolation[C]//Proceedings of the 2015 13th Annual International Conference on Mobile Systems, Applications, and Services. New York: ACM, 2015: 287-300.

参数	数值
总容量/GB	64
保留的自由数据块/%	10
垃圾回收策略	Greedy
每个盘片的块	1024
每个块上的页	64
页大小/kB	4

应用名称	重复率/%	IOPS	大小/MB	写比率/%
Wechat	26.6	4.6	345.5	90.41
QQ	26.6	1.5	84.4	96.40
Weibo	35.4	1.4	255.4	77.53
Tou tiao	23.8	2.0	64.9	86.48
Baidu Map	17.9	4.3	311.7	94.22
Chrome	27.0	3.0	191.5	90.41

移动闪存的重复数据删除技术

Deduplication technology for the mobile flash

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 23

相关文章 4

Metrics

本文评价

推荐阅读 0

[1]	王丹, 孙蓉, 韩辉. 两类多重置换阵列码的构造方法[J]. 西安电子科技大学学报, 2024, 51(6): 84-90.
[2]	林耿豪,周子集,唐鑫,周艺腾,钟宇琪,齐天旸. 采用随机块附加策略的云数据安全去重方法[J]. 西安电子科技大学学报, 2023, 50(5): 212-228.
[3]	何雅萍,贺玉成,周林. 纠正稳定和非稳定突发擦除错误的置换码[J]. 西安电子科技大学学报, 2021, 48(3): 49-55.
[4]	慕建君;赵鹏;焦晓鹏. 一种纠单个闪存单元移位错误码的译码方法[J]. 西安电子科技大学学报, 2016, 43(6): 51-55+121.