Abstract
NAND flash memory is widely used in various systems, ranging from real-time embedded systems to enterprise server systems. Because the flash memory has erase-before-write characteristics, we need flash-memory management methods, i.e., address translation and garbage collection. In particular, garbage collection (GC) incurs long-tail latency, e.g., 100 times higher latency than the average latency at the 99th percentile. Thus, real-time and quality-critical systems fail to meet the given requirements such as deadline and QoS constraints. In this study, we propose a novel method of GC based on reinforcement learning. The objective is to reduce the long-tail latency by exploiting the idle time in the storage system. To improve the efficiency of the reinforcement learning-assisted GC scheme, we present new optimization methods that exploit fine-grained GC to further reduce the long-tail latency. The experimental results with real workloads show that our technique significantly reduces the long-tail latency by 29--36% at the 99.99th percentile compared to state-of-the-art schemes.
- Qi Zhang, Xuandong Li, Linzhang Wang, Tian Zhang, Yi Wang, and Zili Shao. 2015. Lazy-RTGC: A real-time lazy garbage collection mechanism with jointly optimizing average and worst performance for NAND flash memory storage systems. ACM Trans. Des. Autom. Electron. Syst. 20, 3, Article 43 (June 2015), 32 pages. Google Scholar
Digital Library
- Zhiwei Qin, Yi Wang, Duo Liu, and Zili Shao. 2012. Real-time flash translation layer for NAND flash memory storage systems. In Symp. IEEE 18th Real Time and Embedded Technology and Application. 35--44. Google Scholar
Digital Library
- Youngjae Kim, Brendan Tauras, Aayush Gupta, and Bhuvan Urgaonkar. 2009. FlashSim: A simulator for NAND flash-based solid-state drives. In Proceeding. 1st International Conference in Advances in System Simulation (SIMUL). 125--131. Google Scholar
Digital Library
- Yi Wang, Zhiwei Qin, Renhai Chen, Zili Shao, Qixin Wang, Shuai Li, and Laurence T. Yang. 2016. A real-time flash translation layer for NAND flash memory storage systems. IEEE Trans. on Multi-scale Computer Systems. 2. 1. 17--29. Google Scholar
Digital Library
- Siddharth Choudhuri and Tony Givargis. 2008. Deterministic service guarantees for nand flash using partial block cleaning. In Proceedings of the 6th IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS '08). ACM, New York, NY, USA, 19--24. Google Scholar
Digital Library
- Hsin-Yu Chang, Chien-Chung Ho, Yuan-Hao Chang, Yu-Ming Chang, and Tei-Wei Kuo. 2016. How to enable software isolation and boost system performance with sub-block erase over 3D flash memory. In Proceedings of the Eleventh IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis (CODES'16). ACM, New York, NY, USA, Article 6, 10 pages. Google Scholar
Digital Library
- Tseng-Yi Chen, Yuan-Hao Chang, Chien-Chung Ho, and Shuo-Han Chen. 2016. Enabling sub-blocks erase management to boost the performance of 3D NAND flash memory. In Proceedings of the 53rd Annual Design Automation Conference (DAC'16). ACM, New York, NY, USA, Article 92, 6 pages. Google Scholar
Digital Library
- Chulbum Kim, Ji-Ho Cho, Woopyo Jeong, Il-han Park, Hyun-Wook Park, Doo-Hyun Kim, Daewoon Kang, Sunghoon Lee, Ji-Sang Lee, Wontae Kim, Jiyoon Park, Yang-lo Ahn, Jiyoung Lee, Jong-hoon Lee, Seungbum Kim, Hyun-Jun Yoon, Jaedoeg Yu, Nayoung Choi, Yelim Kwon, Nahyun Kim, Hwajun Jang, Jonghoon Park, Seunghwan Song, Yongha Park, Jinbae Bang, Sangki Hong, Byunghoon Jeong, Hyun-Jin Kim, Chunan Lee, Young-Sun Min, Inryul Lee, In-Mo Kim, Sung-Hoon Kim, Dongkyu Yoon, Ki-Sung Kim, Youngdon Choi, Moosung Kim, Hyunggon Kim, Pansuk Kwak, Jeong-Don Ihm, Dae-Seok Byeon, Jin-yub Lee, Ki-Tae Park, and Kye-hyun Kyung. 2017. A 512Gb 3b/cell 64-stacked WL 3D V-NAND flash memory. In Proceeding. 2017 IEEE International Solid-State Circuits Conference (ISSCC). 202--204.Google Scholar
Cross Ref
- Sungdae Choi, Duckju Kim, Sungwook Choi, Byungryul Kim, Sunghyun Jung, Kichang Chun, Namkyeong Kim, Wanseob Lee, Taisik Shin, Hyunjong Jin, Hyunchul Cho, Sunghoon Ahn, Yonghwan Hong, Ingon Yang, Byoungyoung Kim, Pilseon Yoo, Youngdon Jung, Jinwoo Lee, Jaehyeon Shin, Taeyun Kim, Kunwoo Park, and Jinwoong Kim. 2014. A 93.4mm2 64Gb MLC NAND-flash memory with 16nm CMOS technology. In Proceeding. 2014 IEEE International Solid-State Circuits Conference (ISSCC). 328--330.Google Scholar
- Richard S. Sutton and Andrew G. Barto. 1998. Introduction to Reinforcement Learning (1st ed.). MIT Press, Cambridge, MA, USA. Google Scholar
Digital Library
- Aayush Gupta, Youngjae Kim, and Bhuvan Urgaonkar. 2009. DFTL: A flash translation layer employing demand-based selective caching of page-level address mappings. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XIV). ACM, New York, NY, USA, 229--240. Google Scholar
Digital Library
- Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau. 2015. Operating Systems: Three easy pieces. Arpaci-Dusseau Books. LLC.Google Scholar
Digital Library
- Qingsong Wei, Bozhao Gong, Suraj Pathak, Bharadwaj Veeravalli, LingFang Zeng, and Kanzo Okada. 2011. WAFTL: A workload adaptive flash translation layer with data partition. In Proceedings of the IEEE 217th Symposium on Mass Storage Systems and Technologies (MSST). Google Scholar
Digital Library
- Kee-Hoon Jang and Tae Hee Han. 2010. Efficient garbage collection policy and block management method for NAND flash memory. In Proceeding. 2010 2nd International Conference on Mechanical and Electronics Engineering (ICMEE). 327--331.Google Scholar
Cross Ref
- Longzhe Han, Yeonseung Ryu, and Keunsoo Yim. 2006. CATA: A garbage collection scheme for flash memory file systems. In Proceeding. International Conference on Ubiquitous Intelligence and Computing (UIC). 103--112. Google Scholar
Digital Library
- Mingwei Lin and Shuyu Chen. 2013. Efficient and intelligent garbage collection policy for NAND flash-based consumer electronics. IEEE Trans. On Consumer Electronics. 2, 3. 538--543.Google Scholar
- Li-Pin Chang, Tei-Wei Kuo, and Shi-Wu Lo. 2004. Real-time garbage collection for flash-memory storage systems of real-time embedded systems. ACM Trans. Embed. Comput. Syst. 3, 4 (November 2004), 837--863. Google Scholar
Digital Library
- Samsung Electronics. 2014. Samsung V-NAND technology white paper. http://www.samsung.com/us/business/oem-solutions/pdfs/V-NAND_technology_WP.pdf.Google Scholar
- SNIA IOTTA: Storage Networking Industry Association's Input/Output Traces, Tools and Analysis. http://iotta.snia.org.Google Scholar
- Filebench. https://github.com/filebench/filebench/wiki.Google Scholar
- Top considerations for Enterprise SSDs a primer. 2016. http://ingrammicrosystemarchitechs.com/wp-content/uploads/2015/10/White-Paper-Top-Considerations-for-Enterprise-SSDs-WP30.pdf.Google Scholar
- Jeffrey Dean and Luiz Andre Barroso. 2013. The tail at scale. Communication of ACM 56. 74--80. Google Scholar
Digital Library
- Christian Monzio Compagnoni, Akira Goda, Alessandro S. Spinelli, Peter Feeley, Andrea L. Lacaita, and Angelo Visconti. 2017. Reviewing the evolution of the NAND flash technology. In Proceedings of the IEEE. 99, 1--25.Google Scholar
- Rino Micheloni, Seiichi Aritome, and Luca Crippa. 2017. Array architectures for 3-D NAND flash memories. In Proceedings of the IEEE. 99, 1--16.Google Scholar
- Engin Ipek, Onur Mutlu, Jose F. Martinez, and Rich Caruana. 2008. Self-optimizing memory controllers: A reinforcement learning approach. In Proceeding. 2008 IEEE International Symposium on Computer Architecture (ISCA’08). Google Scholar
Digital Library
- Yanzhi Wang, Qing Xie, Ahmed Ammari, and Massoud Pedram. 2011. Deriving a near-optimal power management policy using model-free reinforcement learning and bayesian classification. In Proceedings of the 48th Design Automation Conference (DAC'11). ACM, New York, NY, USA, 41--46. Google Scholar
Digital Library
- Leeor Peled, Shie Mannor, Uri Weiser, and Yoav Etsion. 2015. Semantic locality and context-based prefetching using reinforcement learning. In Proceeding. ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA), Portland. 285--297. Google Scholar
Digital Library
Index Terms
Reinforcement Learning-Assisted Garbage Collection to Mitigate Long-Tail Latency in SSD
Recommendations
Dynamic management of key states for reinforcement learning-assisted garbage collection to reduce long tail latency in SSD
DAC '18: Proceedings of the 55th Annual Design Automation ConferenceGarbage collection (GC) is one of main causes of the long-tail latency problem in storage systems. Long-tail latency due to GC is more than 100 times greater than the average latency at the 99th percentile. Therefore, due to such a long tail latency, ...
Age-based garbage collection
Modern generational garbage collectors look for garbage among the young objects, because they have high mortality; however, these objects include the very youngest objects, which clearly are still live. We introduce new garbage collection algorithms, ...
Garbage Collection Algorithms for NAND Flash Memory Devices -- An Overview
EMS '13: Proceedings of the 2013 European Modelling SymposiumThis paper highlights the necessary garbage collection algorithms which is needed in flash translation layer (FTL) for NAND Flash memory devices such as SATA SSD's (Solid State Devices). Garbage collection is the process of freeing up partially filled ...






Comments