skip to main content
research-article

Telomere: Real-Time NAND Flash Storage

Published:14 January 2022Publication History
Skip Abstract Section

Abstract

Modern solid-state disks achieve high data transfer rates due to their massive internal parallelism. However, out-of-place updates for flash memory incur garbage collection costs when valid data needs to be copied during space reclamation. The root cause of this extra cost is that solid-state disks are not always able to accurately determine data lifetime and group together data that expires before the space needs to be reclaimed. Real-time systems found in autonomous vehicles, industrial control systems, and assembly-line robots store data from hundreds of sensors and often have predictable data lifetimes. These systems require guaranteed high storage bandwidth for read and write operations by mission-critical real-time tasks. In this article, we depart from the traditional block device interface to guarantee the high throughput needed to process large volumes of data. Using data lifetime information from the application layer, our proposed real-time design, called Telomere, is able to intelligently lay out data in NAND flash memory and eliminate valid page copies during garbage collection. Telomere’s real-time admission control is able to guarantee tasks their required read and write operations within their periods. Under randomly generated tasksets containing 500 tasks, Telomere achieves 30% higher throughput with a 5% storage cost compared to pre-existing techniques.

REFERENCES

  1. [1] Ackerman Evan. 2016. Autonomous Mini Rally Car Teaches Itself to Powerslide. Retrieved November 20, 2021 from https://spectrum.ieee.org/cars-that-think/transportation/self-driving/autonomous-mini-rally-car-teaches-itself-to-powerslide.Google ScholarGoogle Scholar
  2. [2] Agrawal Nitin, Prabhakaran Vijayan, Wobber Ted, Davis John D., Manasse Mark, and Panigraphy Rina. 2008. Design tradeoffs for SSD performance. In Proceedings of the USENIX Annual Technical Conference (ATC’08). Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Angelica Amara D.. 2013. Google’s Self-Driving Car Gathers Nearly 1 GB/sec. Retrieved November 20, 2021 from http://www.kurzweilai.net/googles-self-driving-car-gathers-nearly-1-gbsec/.Google ScholarGoogle Scholar
  4. [4] Baker Theodore P.. 1990. A stack-based resource allocation policy for realtime processes. In Proceedings of the Real-Time Systems Symposium (RTSS’90).Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Berger Miriam. 2020. How the world’s beaches are readying for a summer of social distancing. Washington Post. Retrieved November 20, 2021 from https://www.washingtonpost.com/world/2020/06/27/how-worlds-beaches-are-readying-summer-social-distancing/.Google ScholarGoogle Scholar
  6. [6] Bini Enrico and Buttazzo Giorgio C.. 2005. Measuring the performance of schedulability tests. Real-Time Systems 30, 1-2 (2005), 129–154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Bjorling Matias. 2019. From open-channel SSDs to zoned namespaces. In Proceedings of the Linux Storage and Filesystems Conference (Vault’19).Google ScholarGoogle Scholar
  8. [8] Bjorling Matias, Bonnet Philippe, Bouganim Luc, and Dayan Niv. 2013. The necessary death of the block device interface. In Proceedings of the Biennial Conference on Innovative Data Systems Research (CIDR’13).Google ScholarGoogle Scholar
  9. [9] Bjorling Matias, Gonzalez Javier, and Bonnet Philippe. 2017. LightNVM: The Linux open-channel SSD subsystem. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’17). Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Chen Feng, Hou Binbing, and Lee Rubao. 2016. Internal parallelism of flash memory-based solid-state drives. ACM Transactions on Storage 12, 3 (2016), Article 13, 39 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Chen Feng, Lee Rubao, and Zhang Xiaodong. 2011. Essential roles of exploiting internal parallelism of flash memory based solid state drives in high-speed data processing. In Proceedings of the IEEE International Symposium on High Performance Computer Architecture. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Chiang Mei-Ling and Chang Ruei-Chuan. 1999. Cleaning policies in mobile computers using flash memory. Journal of System Software 48, 3 (1999), 213231. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Choudhuri Siddharth and Givargis Tony. 2008. Deterministic service guarantees for NAND flash using partial block cleaning. In Proceedings of the IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Colgrove John, Davis John D., Hayes John, Miller Ethan L., Sandvig Cary, Sears Russell, Tamches Ari, Vachharajani Neil, and Wang Feng. 2015. Purity: Building fast, highly-available enterprise flash storage from commodity components. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’15). Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Hao Mingzhe, Soundararajan Gokul, Kenchammana-Hosekote Deepak, Chien Andrew A., and Gunawi Haryadi S.. 2016. The tail at store: A revelation from millions of hours of disk and SSD deployments. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’16). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Huang Sheng-Min and Chang Li-Pin. 2018. Providing SLO compliance on NVMe SSDs through parallelism reservation. ACM Transactions on Design Automation of Electronic Systems 23, 3 (2018), Article 28, 26 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Kang Jeong-Uk, Hyun Jeeseok, Maeng Hyunjoo, and Cho Sangyeun. 2014. The multi-streamed solid-state drive. In Proceedings of the USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage’14). Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Kang Yangwook, Yang Jingpei, and Miller Ethan L.. 2011. SCM: An efficient interface for storage class memories. In Proceedings of IEEE Symposium on Mass Storage Systems and Technologies (MSST’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Kawaguchi Atsuo, Nishioka Shingo, and Motoda Hiroshi. 1995. A flash-memory based file system. In Proceedings of the USENIX Technical Conference on UNIX and Advanced Computing Systems. 155164. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Kim Jaeho, Lee Donghee, and Noh Sam H.. 2015. Towards SLO complying SSDs through OPS isolation. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’15). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Kim Sungchan, Oh Hyunok, Park Chanik, Cho Sangyeun, and Lee Sang-Won. 2011. Fast, energy efficient scan inside flash memory SSDs. In Proceedings of the International Workshop on Accelerating Data Management Systems.Google ScholarGoogle Scholar
  22. [22] Lu Youyou, Shu Jiwu, and Zheng Weimin. 2013. Extending the lifetime of flash-based storage through reducing write amplification from file systems. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’13). Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Luo Yixin, Cai Yu, Ghose Saugata, Choi Jongmoo, and Mutlu Onur. 2015. WARM: Improving NAND flash memory lifetime with write-hotness aware retention management. In Proceedings of the Conference on Mass Storage Systems and Technology (MSST’15).Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Ma Dongzhe, Feng Jianhua, and Li Guoliang. 2014. A survey of address translation technologies for flash memories. ACM Computing Surveys 46, 3 (2014), 36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Micron. 2005. Micron Technical Report: Small-Block vs. Large-Block NAND Flash Devices. Technical Report TN-29-07. Micron.Google ScholarGoogle Scholar
  26. [26] Micron. 2017. Micron Reveals Critical Technologies for Autonomous Vehicles. Retrieved November 20, 2021 from https://investors.micron.com/news-releases/news-release-details/micron-reveals-critical-technologies-autonomous-vehicles.Google ScholarGoogle Scholar
  27. [27] Micron. 2018. NAND Flash Die—128Gb Die: X8 300mm MLC MT29F128G08CBECB. Retrieved November 20, 2021 from https://prod.micron.com/media/documents/products/data-sheet/nand-flash/die/l95b_die_128gb_nand.pdf.Google ScholarGoogle Scholar
  28. [28] Missimer Katherine and West Richard. 2018. Partitioned real-time NAND flash storage. In Proceedings of the Real-Time Systems Symposium (RTSS’18).Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Mohan Vidyabhushan, Siddiqua Taniya, Gurumurthi Sudhanva, and Stan Mircea R.. 2010. How I learned to stop worrying and love flash endurance. In Proceedings of the USENIX Conference on Hot Topics in Storage and File Systems (FAST’10). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Oh HakJune. 2013. Single Controller 4/8TB SSD. Retrieved November 20, 2021 from https://www.flashmemory summit.com/English/Collaterals/Proceedings/2013/20130815_301A_Oh.pdf.Google ScholarGoogle Scholar
  31. [31] Park Dongchul and Du David H. C.. 2011. Hot data identification for flash-based storage systems using multiple Bloom filters. In Proceedings of the Conference on Massive Storage Systems and Technology (MSST’11).Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Pletka Roman, Koltsidas Ioannis, Ioannou Nikolas, Tomic Sasa, Papandreou Nikolaos, Parnell Thomas, Pozidis Haralampos, Fry Aaron, and Fisher Tim. 2018. Management of next-generation NAND flash to achieve enterprise-level endurance and latency targets. ACM Transactions on Storage 14, 4 (2018), Article 33, 25 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Pletka Roman A. and Tomic Sasa. 2016. Health-binning: Maximizing the performance and the endurance of consumer-level NAND flash. In Proceedings of the 9th ACM International Conference on Systems and Storage (SYSTOR’16). Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Qin Zhiwei, Wang Yi, Liu Duo, and Shao Zili. 2012. Real-time flash translation layer for NAND flash memory storage systems. In Proceedings of the IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Rosenblum Mendel and Ousterhout John K.. 1992. The design and implementation of a log-structured file system. ACM Transactions on Computer Systems 10, 1 (1992), 2652. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Ruffo Gustavo Henrique. 2019. Tesla Cars Have a Memory Problem That May Cost You a Lot to Repair. https://insideevs.com/news/376037/tesla-mcu-emmc-memory-issue/.Google ScholarGoogle Scholar
  37. [37] SanDisk. 2018. iNAND Automotive Embedded Flash Drives. Retrieved November 20, 2021 from https://www.sandisk.com/oem-design/automotive/inand.Google ScholarGoogle Scholar
  38. [38] Skourtis Dimitris, Achlioptas Dimitris, Watkins Noah, Maltzahn Carlos, and Brandt Scott. 2014. Flash on rails: Consistent flash performance through redundancy. In Proceedings of the USENIX Annual Technical Conference (ATC’14). Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Song Yong Ho, Jung Sanghyuk, Lee Sang-Won, and Kim Jin-Soo. 2014. Cosmos OpenSSD: A PCIe-Based Open Source SSD Platform. Retrieved November 20, 2021 from http://www.flashmemorysummit.com/English/Collaterals/Proceedings/\2014/20140807_301B_Song.pdf.Google ScholarGoogle Scholar
  40. [40] Spires Josh. 2020. How Drones Have Helped Fight COVID-19—And Become More Mainstream. Retrieved November 20, 2021 from https://dronedj.com/2020/06/04/.Google ScholarGoogle Scholar
  41. [41] Stoica Radu and Ailamaki Anastasia. 2013. Improving flash write performance by using update frequency. In Proceedings of the International Conference on Very Large Data Bases (VLDB’13).Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Digital Western. 2019. Zoned Storage. Retrieved November 20, 2021 from http://zonedstorage.io.Google ScholarGoogle Scholar
  43. [43] Wu Michael and Zwaenepoel Willy. 1994. eNVy: A non-volatile, main memory storage system. ACM SIGPLAN Notices 29, 11 (1994), 86–97. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Yan Shiqin, Li Huaicheng, Hao Mingzhe, Tong Michael Hao, Sundararaman Swaminathan, Chien Andrew A., and Gunawi Haryadi S.. 2017. Tiny-tail flash: Near-perfect elimination of garbage collection tail latencies in NAND SSDs. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’17). Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Yang Jingpei, Pandurangan Rajinikanth, Choi Changho, and Balakrishnan Vijay. 2017. AutoStream: Automatic stream management for multi-streamed SSDs. In Proceedings of the ACM International Conference on Systems and Storage (SYSTOR’17). Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Yang Jingpei, Plasson Ned, Gillis Greg, Talagala Nisha, and Sundararaman Swaminathan. 2014. Don’t stack your log on my log. In Proceedings of the Workshop on Interactions of NVM/Flash with Operating Systems and Workloads (INFLOW’14).Google ScholarGoogle Scholar
  47. [47] Zhang Qi, Li Xuandong, Wang Linzhang, Zhang Tian, Wang Yi, and Shao Zili. 2015. Optimizing deterministic garbage collection in NAND flash storage systems. In Proceedings of the Real-Time and Embedded Technology and Applications Symposium (RTAS’15). IEEE, Los Alamitos, CA.Google ScholarGoogle ScholarCross RefCross Ref
  48. [48] Zhao K., Zhao W., Sun H., Zhang T., Zhang X., and Zheng N.. 2013. LDPC-in-SSD: Making advanced error correction codes work effectively in solid state drives. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’13). Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] West R., Li Y., Missimer E., and Danish M.. 2016. A virtualized separation Kernel for mixed criticality systems. ACM Transactions on Computer Systems 34 (2016).Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Telomere: Real-Time NAND Flash Storage

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Embedded Computing Systems
      ACM Transactions on Embedded Computing Systems  Volume 21, Issue 1
      January 2022
      288 pages
      ISSN:1539-9087
      EISSN:1558-3465
      DOI:10.1145/3505211
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 14 January 2022
      • Accepted: 1 July 2021
      • Revised: 1 June 2021
      • Received: 1 January 2021
      Published in tecs Volume 21, Issue 1

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!