skip to main content
research-article

Reinforcement Learning-Assisted Garbage Collection to Mitigate Long-Tail Latency in SSD

Authors Info & Claims
Published:27 September 2017Publication History
Skip Abstract Section

Abstract

NAND flash memory is widely used in various systems, ranging from real-time embedded systems to enterprise server systems. Because the flash memory has erase-before-write characteristics, we need flash-memory management methods, i.e., address translation and garbage collection. In particular, garbage collection (GC) incurs long-tail latency, e.g., 100 times higher latency than the average latency at the 99th percentile. Thus, real-time and quality-critical systems fail to meet the given requirements such as deadline and QoS constraints. In this study, we propose a novel method of GC based on reinforcement learning. The objective is to reduce the long-tail latency by exploiting the idle time in the storage system. To improve the efficiency of the reinforcement learning-assisted GC scheme, we present new optimization methods that exploit fine-grained GC to further reduce the long-tail latency. The experimental results with real workloads show that our technique significantly reduces the long-tail latency by 29--36% at the 99.99th percentile compared to state-of-the-art schemes.

References

  1. Qi Zhang, Xuandong Li, Linzhang Wang, Tian Zhang, Yi Wang, and Zili Shao. 2015. Lazy-RTGC: A real-time lazy garbage collection mechanism with jointly optimizing average and worst performance for NAND flash memory storage systems. ACM Trans. Des. Autom. Electron. Syst. 20, 3, Article 43 (June 2015), 32 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Zhiwei Qin, Yi Wang, Duo Liu, and Zili Shao. 2012. Real-time flash translation layer for NAND flash memory storage systems. In Symp. IEEE 18th Real Time and Embedded Technology and Application. 35--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Youngjae Kim, Brendan Tauras, Aayush Gupta, and Bhuvan Urgaonkar. 2009. FlashSim: A simulator for NAND flash-based solid-state drives. In Proceeding. 1st International Conference in Advances in System Simulation (SIMUL). 125--131. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Yi Wang, Zhiwei Qin, Renhai Chen, Zili Shao, Qixin Wang, Shuai Li, and Laurence T. Yang. 2016. A real-time flash translation layer for NAND flash memory storage systems. IEEE Trans. on Multi-scale Computer Systems. 2. 1. 17--29. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Siddharth Choudhuri and Tony Givargis. 2008. Deterministic service guarantees for nand flash using partial block cleaning. In Proceedings of the 6th IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS '08). ACM, New York, NY, USA, 19--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Hsin-Yu Chang, Chien-Chung Ho, Yuan-Hao Chang, Yu-Ming Chang, and Tei-Wei Kuo. 2016. How to enable software isolation and boost system performance with sub-block erase over 3D flash memory. In Proceedings of the Eleventh IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis (CODES'16). ACM, New York, NY, USA, Article 6, 10 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Tseng-Yi Chen, Yuan-Hao Chang, Chien-Chung Ho, and Shuo-Han Chen. 2016. Enabling sub-blocks erase management to boost the performance of 3D NAND flash memory. In Proceedings of the 53rd Annual Design Automation Conference (DAC'16). ACM, New York, NY, USA, Article 92, 6 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Chulbum Kim, Ji-Ho Cho, Woopyo Jeong, Il-han Park, Hyun-Wook Park, Doo-Hyun Kim, Daewoon Kang, Sunghoon Lee, Ji-Sang Lee, Wontae Kim, Jiyoon Park, Yang-lo Ahn, Jiyoung Lee, Jong-hoon Lee, Seungbum Kim, Hyun-Jun Yoon, Jaedoeg Yu, Nayoung Choi, Yelim Kwon, Nahyun Kim, Hwajun Jang, Jonghoon Park, Seunghwan Song, Yongha Park, Jinbae Bang, Sangki Hong, Byunghoon Jeong, Hyun-Jin Kim, Chunan Lee, Young-Sun Min, Inryul Lee, In-Mo Kim, Sung-Hoon Kim, Dongkyu Yoon, Ki-Sung Kim, Youngdon Choi, Moosung Kim, Hyunggon Kim, Pansuk Kwak, Jeong-Don Ihm, Dae-Seok Byeon, Jin-yub Lee, Ki-Tae Park, and Kye-hyun Kyung. 2017. A 512Gb 3b/cell 64-stacked WL 3D V-NAND flash memory. In Proceeding. 2017 IEEE International Solid-State Circuits Conference (ISSCC). 202--204.Google ScholarGoogle ScholarCross RefCross Ref
  9. Sungdae Choi, Duckju Kim, Sungwook Choi, Byungryul Kim, Sunghyun Jung, Kichang Chun, Namkyeong Kim, Wanseob Lee, Taisik Shin, Hyunjong Jin, Hyunchul Cho, Sunghoon Ahn, Yonghwan Hong, Ingon Yang, Byoungyoung Kim, Pilseon Yoo, Youngdon Jung, Jinwoo Lee, Jaehyeon Shin, Taeyun Kim, Kunwoo Park, and Jinwoong Kim. 2014. A 93.4mm2 64Gb MLC NAND-flash memory with 16nm CMOS technology. In Proceeding. 2014 IEEE International Solid-State Circuits Conference (ISSCC). 328--330.Google ScholarGoogle Scholar
  10. Richard S. Sutton and Andrew G. Barto. 1998. Introduction to Reinforcement Learning (1st ed.). MIT Press, Cambridge, MA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Aayush Gupta, Youngjae Kim, and Bhuvan Urgaonkar. 2009. DFTL: A flash translation layer employing demand-based selective caching of page-level address mappings. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XIV). ACM, New York, NY, USA, 229--240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau. 2015. Operating Systems: Three easy pieces. Arpaci-Dusseau Books. LLC.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Qingsong Wei, Bozhao Gong, Suraj Pathak, Bharadwaj Veeravalli, LingFang Zeng, and Kanzo Okada. 2011. WAFTL: A workload adaptive flash translation layer with data partition. In Proceedings of the IEEE 217th Symposium on Mass Storage Systems and Technologies (MSST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Kee-Hoon Jang and Tae Hee Han. 2010. Efficient garbage collection policy and block management method for NAND flash memory. In Proceeding. 2010 2nd International Conference on Mechanical and Electronics Engineering (ICMEE). 327--331.Google ScholarGoogle ScholarCross RefCross Ref
  15. Longzhe Han, Yeonseung Ryu, and Keunsoo Yim. 2006. CATA: A garbage collection scheme for flash memory file systems. In Proceeding. International Conference on Ubiquitous Intelligence and Computing (UIC). 103--112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mingwei Lin and Shuyu Chen. 2013. Efficient and intelligent garbage collection policy for NAND flash-based consumer electronics. IEEE Trans. On Consumer Electronics. 2, 3. 538--543.Google ScholarGoogle Scholar
  17. Li-Pin Chang, Tei-Wei Kuo, and Shi-Wu Lo. 2004. Real-time garbage collection for flash-memory storage systems of real-time embedded systems. ACM Trans. Embed. Comput. Syst. 3, 4 (November 2004), 837--863. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Samsung Electronics. 2014. Samsung V-NAND technology white paper. http://www.samsung.com/us/business/oem-solutions/pdfs/V-NAND_technology_WP.pdf.Google ScholarGoogle Scholar
  19. SNIA IOTTA: Storage Networking Industry Association's Input/Output Traces, Tools and Analysis. http://iotta.snia.org.Google ScholarGoogle Scholar
  20. Filebench. https://github.com/filebench/filebench/wiki.Google ScholarGoogle Scholar
  21. Top considerations for Enterprise SSDs a primer. 2016. http://ingrammicrosystemarchitechs.com/wp-content/uploads/2015/10/White-Paper-Top-Considerations-for-Enterprise-SSDs-WP30.pdf.Google ScholarGoogle Scholar
  22. Jeffrey Dean and Luiz Andre Barroso. 2013. The tail at scale. Communication of ACM 56. 74--80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Christian Monzio Compagnoni, Akira Goda, Alessandro S. Spinelli, Peter Feeley, Andrea L. Lacaita, and Angelo Visconti. 2017. Reviewing the evolution of the NAND flash technology. In Proceedings of the IEEE. 99, 1--25.Google ScholarGoogle Scholar
  24. Rino Micheloni, Seiichi Aritome, and Luca Crippa. 2017. Array architectures for 3-D NAND flash memories. In Proceedings of the IEEE. 99, 1--16.Google ScholarGoogle Scholar
  25. Engin Ipek, Onur Mutlu, Jose F. Martinez, and Rich Caruana. 2008. Self-optimizing memory controllers: A reinforcement learning approach. In Proceeding. 2008 IEEE International Symposium on Computer Architecture (ISCA’08). Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Yanzhi Wang, Qing Xie, Ahmed Ammari, and Massoud Pedram. 2011. Deriving a near-optimal power management policy using model-free reinforcement learning and bayesian classification. In Proceedings of the 48th Design Automation Conference (DAC'11). ACM, New York, NY, USA, 41--46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Leeor Peled, Shie Mannor, Uri Weiser, and Yoav Etsion. 2015. Semantic locality and context-based prefetching using reinforcement learning. In Proceeding. ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA), Portland. 285--297. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Reinforcement Learning-Assisted Garbage Collection to Mitigate Long-Tail Latency in SSD

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!