skip to main content
research-article

HIL: A Framework for Compositional FTL Development and Provably-Correct Crash Recovery

Published:04 December 2018Publication History
Skip Abstract Section

Abstract

We present a framework called Hierarchically Interacting Logs (HIL) for constructing Flash Translation Layers (FTLs). The main goal of the HIL framework is to heal the Achilles heel —the crash recovery—of FTLs (hence, its name). Nonetheless, the framework itself is general enough to encompass not only block-mapped and page-mapped FTLs but also many of their variants, including hybrid ones, because of its compositional nature.

Crash recovery within the HIL framework proceeds in two phases: structural recovery and functional recovery. During the structural recovery, residual effects due to program operations ongoing at the time of the crash are eliminated in an atomic manner using shadow paging. During the functional recovery, operations that would have been performed if there had been no crash are replayed in a redo-only fashion. Both phases operate in an idempotent manner, preventing repeated crashes during recovery from causing any additional problems.

We demonstrate the practicality of the proposed HIL framework by implementing a prototype and showing that its performance during normal execution and also during crash recovery is at least as good as those of state-of-the-art SSDs.

References

  1. Nitin Agrawal, Vijayan Prabhakaran, Ted Wobber, John D. Davis, Mark Manasse, and Rina Panigrahy. 2008. Design tradeoffs for SSD performance. In Proceedings of the USENIX 2008 Annual Technical Conference (ATC’08). USENIX Association, Boston, MA, 57--70. Retrieved from http://dl.acm.org/citation.cfm?id=1404014.1404019. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Seiichi Aritome. 2015. NAND Flash Memory Technologies. John Wiley 8 Sons, Inc., Hoboken, NJ.Google ScholarGoogle Scholar
  3. Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau. 2015. Three Easy Pieces. Arpaci-Dusseau Books.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Philip A. Bernstein, Vassco Hadzilacos, and Nathan Goodman. 1987. Concurrency Control and Recovery in Database Systems. Addison-Wesley Longman Publishing Co., Inc., Boston, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Nathan Binkert, Bradford Beckmann, Gabriel Black, Steven K. Reinhardt, Ali Saidi, Arkaprava Basu, Joel Hestness, Derek R. Hower, Tushar Krishna, Somayeh Sardashti, Rathijit Sen, Korey Sewell, Muhammad Shoaib, Nilay Vaish, Mark D. Hill, and David A. Wood. 2011. The Gem5 simulator. ACM SIGARCH Comput. Architect. News 39, 2 (Aug. 2011), 1--7. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Andrew Birrell, Michael Isard, Chuck Thacker, and Ted Wobber. 2007. A design for high-performance flash disks. ACM SIGOPS Operat. Syst. Rev. 41, 2 (Apr. 2007), 88--93. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Yu Cai, Saugata Ghose, Yixin Luo, Ken Mai, Onur Mutlu, and Erich F. Haratsch. 2017. Vulnerabilities in MLC NAND flash memory programming: Experimental analysis, exploits, and mitigation techniques. In Proceedings of the 23rd IEEE International Symposium on High Performance Computer Architecture (HPCA’17). Austin, TX, 49--60. Retrieved fromGoogle ScholarGoogle Scholar
  8. Yu-Ming Chang, Ping-Hsien Lin, Ye-Jyun Lin, Tai-Chun Kuo, Yuan-Hao Chang, Yung-Chun Li, Hsiang-Pang Li, and K. C. Wang. 2016. An efficient sudden-power-off-recovery design with guaranteed booting time for solid-state drives. In Proceedings of the 8th IEEE International Memory Workshop (IMW’16). Paris, France, 1--4. Retrieved fromGoogle ScholarGoogle Scholar
  9. Mei-Ling Chiang and Ruei-Chuan Chang. 1999. Cleaning policies in mobile computers using flash memory. J. Syst. Softw. 48, 3 (Nov. 1999), 213--231. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Niv Dayan, Philippe Bonnet, and Stratos Idreos. 2016. GeckoFTL: Scalable flash translation techniques for very large flash devices. In Proceedings of the 2016 International Conference on Management of Data (SIGMOD’16). ACM, San Francisco, CA, 327--342. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Eran Gal and Sivan Toledo. 2005. Algorithms and data structures for flash memories. Comput. Surveys 37, 2 (June 2005), 138--163. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Gregory R. Ganger, Marshall K. McKusick, Craig A. N. Soules, and Yale N. Patt. 2000. Soft updates: A solution to the metadata update problem in file systems. ACM Trans. Comput. Syst. 18, 2 (May 2000), 127--153. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jim Gray, Paul McJones, Mike Blasgen, Bruce Lindsay, Raymond Lorie, Tom Price, Franco Putzolu, and Irving Traiger. 1981. The recovery manager of the System R database manager. Comput. Surveys 13, 2 (June 1981), 223--242. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Laura M. Grupp, John D. Davis, and Steven Swanson. 2012. The bleak future of NAND flash memory. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12). USENIX Association, San Jose, CA, 17--24. Retrieved from http://dl.acm.org/citation.cfm?id=2208461.2208463. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Aayush Gupta, Youngjae Kim, and Bhuvan Urgaonkar. 2009. DFTL: A flash translation layer employing demand-based selective caching of page-level address mappings. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’09). ACM, Washington, DC, 229--240. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Dave Hitz, James Lau, and Michael Malcolm. 1994. File system design for an NFS file server appliance. In Proceedings of the USENIX Winter 1994 Technical Conference (WTEC’94). USENIX Association, San Francisco, CA, 19--19. Retrieved from http://dl.acm.org/citation.cfm?id=1267074.1267093. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Po-Chun Huang, Yuan-Hao Chang, and Tei-Wei Kuo. 2013. An index-based management scheme with adaptive caching for huge-scale low-cost embedded flash storages. ACM Transactions on Design Automation of Electronic Systems (TODAES) 18, 4, Article 56 (Oct. 2013). Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Intel Corporation 2015. Intel Solid-State Drive DC P3608 Series Product Specification. Intel Corporation. Retrieved from https://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-dc-p3608-spec.pdf.Google ScholarGoogle Scholar
  19. Intel Corporation and Seagate Technology. 2003. Serial ATA Native Command Queuing. White Paper. Retrieved from http://www.seagate.com/docs/pdf/whitepaper/D2c_tech_paper_intc-stx_sata_ncq.pdf.Google ScholarGoogle Scholar
  20. Myoungsoo Jung and Mahmut Kandemir. 2012. An evaluation of different page allocation strategies on high-speed SSDs. In Proceedings of the 4th Workshop on Hot Topics in Storage and File Systems (HotStorage’12). Boston, MA, 1--5. Retrieved from https://www.usenix.org/system/files/conference/hotstorage12/hotstorage12-final55.pdf. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Jeong-Uk Kang, Heeseung Jo, Jin-Soo Kim, and Joonwon Lee. 2006. A superblock-based flash translation layer for NAND flash memory. In Proceedings of the 6th ACM 8 IEEE International Conference on Embedded Software (EMSOFT’06). ACM, Seoul, Korea, 161--170. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Atsuo Kawaguchi, Shingo Nishioka, and Hiroshi Motoda. 1995. A flash-memory based file system. In Proceedings of the USENIX 1995 Technical Conference (TCON’95). USENIX Association, New Orleans, LA, 13--13. Retrieved from http://dl.acm.org/citation.cfm?id=1267411.1267424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Dongwook Kim, Yujip Won, Jaehyuk Cha, Sungroh Yoon, Jongmoo Choi, and Sooyong Kang. 2016. Exploiting compression-induced internal fragmentation for power-off recovery in SSD. IEEE Trans. Comput. 65, 6 (June 2016), 1720--1733. Retrieved fromGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  24. Hong S. Kim, Eyee H. Nam, Ji H. Yun, Sheayun Lee, and Sang L. Min. 2017. P-BMS: A bad block management scheme in parallelized flash memory storage devices. ACM Trans. Embed. Comput. Syst. 16, 5s, Article 140 (Sept. 2017). Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jesung Kim, Jong M. Kim, Sam H. Noh, Sang L. Min, and Yookun Cho. 2002. A space-efficient flash translation layer for CompactFlash systems. IEEE Trans. Consum. Electron. 48, 2 (May 2002), 366--375. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Hunki Kwon, Eunsam Kim, Jongmoo Choi, Donghee Lee, and Sam H. Noh. 2010. Janus-FTL: Finding the optimal point on the spectrum between page and block mapping schemes. In Proceedings of the 10th ACM International Conference on Embedded Software (EMSOFT’10). ACM, Scottsdale, AZ, 169--178. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Jongmin Lee, Yongseok Oh, Hunki Kwon, Jongmoo Choi, Donghee Lee, and Sam H. Noh. 2013. TinyFTL: An FTL architecture for flash memory cards with scarce resources. In Proceedings of the 4th Asia-Pacific Workshop on Systems (APSys’13). Singapore, Singapore, 16:1--16:5. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Sungjin Lee, Dongkun Shin, Young-Jin Kim, and Jihong Kim. 2008. LAST: Locality-aware sector translation for NAND flash memory-based storage systems. ACM SIGOPS Operat. Syst. Rev. 42, 6 (Oct. 2008), 36--42. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Sang-Won Lee, Dong-Joo Park, Tae-Sun Chung, Dong-Ho Lee, Sangwon Park, and Ha-Joo Song. 2007. A log buffer-based flash translation layer using fully-associative sector translation. ACM Trans. Embed. Comput. Syst. 6, 3, Article 18 (July 2007). Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Yong-Goo Lee, Dawoon Jung, Dongwon Kang, and Jin-Soo Kim. 2008. μ-FTL: A memory-efficient flash translation layer supporting multiple mapping granularities. In Proceedings of the 8th ACM International Conference on Embedded Software (EMSOFT’08). ACM, Atlanta, GA, 21--30. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Youyou Lu, Jiwu Shu, Jia Guo, Shuai Li, and Onur Mutlu. 2013. LightTx: A lightweight transactional design in flash-based SSDs to support flexible transactions. In Proceedings of the 31st IEEE International Conference on Computer Design (ICCD’13). Asheville, NC, 115--122. Retrieved fromGoogle ScholarGoogle ScholarCross RefCross Ref
  32. Dongzhe Ma, Jianhua Feng, and Guoliang Li. 2011. LazyFTL: A page-level flash translation layer optimized for NAND flash memory. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data (SIGMOD’11). ACM, Athens, Greece, 1--12. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. C. Mohan, Don Haderle, Bruce Lindsay, Hamid Pirahesh, and Peter Schwarz. 1992. ARIES: A transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging. ACM Trans. Database Syst. 17, 1 (Mar. 1992), 94--162. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Open Source Development Lab. {n.d.}. Iometer. Open Source Development Lab. Retrieved from http://iometer.org/.Google ScholarGoogle Scholar
  35. Xiangyong Ouyang, David Nellans, Robert Wipfel, David Flynn, and Dhabaleswar K. Panda. 2011. Beyond block I/O: Rethinking traditional storage primitives. In Proceedings of the 17th International Symposium on High Performance Computer Architecture (HPCA’11). IEEE Computer Society, San Antonio, TX, 301--311. Retrieved from http://dl.acm.org/citation.cfm?id=2014698.2014867. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Chanik Park, Wonmoon Cheon, Jeonguk Kang, Kangho Roh, Wonhee Cho, and Jin-Soo Kim. 2008. A reconfigurable FTL (flash translation layer) architecture for NAND flash-based applications. ACM Trans. Embed. Comput. Syst. 7, 4, Article 38 (Aug. 2008). Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Vijayan Prabhakaran, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2005. Analysis and evolution of journaling file systems. In Proceedings of the USENIX Annual Technical Conference (ATEC’05). USENIX Association, Anaheim, CA, 8--8. Retrieved from http://dl.acm.org/citation.cfm?id=1247360.1247368. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Vijayan Prabhakaran, Thomas L. Rodeheffer, and Lidong Zhou. 2008. Transactional flash. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI’08). USENIX Association, San Diego, CA, 147--160. Retrieved from http://dl.acm.org/citation.cfm?id=1855741.1855752. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Betty Prince. 2014. Vertical 3D Memory Technologies. John Wiley 8 Sons, Inc., Hoboken, NJ. Retrieved from https://books.google.co.kr/books?id=s9NLBAAAQBAJ.Google ScholarGoogle Scholar
  40. Raghu Ramakrishnan and Johannes Gehrke. 2000. Database Management Systems. McGraw-Hill. Retrieved from https://books.google.co.kr/books?id=3MxQAAAAMAAJ. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Andrew M. Rogers, Aman Kansal, and Sachin C. Patel. 2010. Rapid Crash Recovery for Flash Storage. Retrieved from https://www.google.co.kr/patents/US7818610US Patent 7,818,610 B2.Google ScholarGoogle Scholar
  42. Samsung Electronics Co., Ltd. 2015. PM1725 NVMe PCIe SSD Product Specification. Samsung Electronics Co., Ltd. Retrieved from http://www.samsung.com/semiconductor/global/file/insight/2015/11/pm1725-ProdOverview-2015-0.pdf.Google ScholarGoogle Scholar
  43. Yoon J. Seong, Eyee H. Nam, Jin H. Yoon, Hongseok Kim, Jin-yong Choi, Sookwan Lee, Young H. Bae, Jaejin Lee, Yookun Cho, and Sang L. Min. 2010. Hydra: A block-mapped parallel flash memory solid-state disk architecture. IEEE Trans. Comput. 59, 7 (July 2010), 905--921. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Hung-Wei Tseng, Laura Grupp, and Steven Swanson. 2011. Understanding the impact of power loss on flash memory. In Proceedings of the 48th Design Automation Conference (DAC’11). ACM, San Diego, CA, 35--40. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Wikipedia. 2010. Write Amplification. Retrieved from https://en.wikipedia.org/wiki/Write_amplification.Google ScholarGoogle Scholar
  46. Guanying Wu and Xubin He. 2012. Reducing SSD read latency via NAND flash program and erase suspension. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12). USENIX Association, San Jose, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Michael Wu and Willy Zwaenepoel. 1994. eNVy: A non-volatile, main memory storage system. In Proceedings of the 6th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’94). ACM, San Jose, CA, 86--97. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Jin H. Yoon, Eyee H. Nam, Yoon J. Seong, Hongseok Kim, Bryan S. Kim, Sang L. Min, and Yookun Cho. 2008. Chameleon: A high performance flash/FRAM hybrid solid-state disk architecture. IEEE Comput. Architect. Lett. 7, 1 (Jan. 2008), 17--20. Retrieved from Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. HIL: A Framework for Compositional FTL Development and Provably-Correct Crash Recovery

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Storage
          ACM Transactions on Storage  Volume 14, Issue 4
          Special Section on Systor 2017 and Regular Papers
          November 2018
          175 pages
          ISSN:1553-3077
          EISSN:1553-3093
          DOI:10.1145/3297750
          • Editor:
          • Sam H. Noh
          Issue’s Table of Contents

          Copyright © 2018 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 4 December 2018
          • Accepted: 1 September 2018
          • Revised: 1 July 2018
          • Received: 1 February 2018
          Published in tos Volume 14, Issue 4

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!