skip to main content
research-article

Accelerating File System Metadata Access with Byte-Addressable Nonvolatile Memory

Authors Info & Claims
Published:24 July 2015Publication History
Skip Abstract Section

Abstract

File system performance is dominated by small and frequent metadata access. Metadata is stored as blocks on the hard disk drive. Partial metadata update results in whole-block read or write, which significantly amplifies disk I/O. Furthermore, a huge performance gap between the CPU and disk aggravates this problem. In this article, a file system metadata accelerator (referred to as FSMAC) is proposed to optimize metadata access by efficiently exploiting the persistency and byte-addressability of Nonvolatile Memory (NVM). The FSMAC decouples data and metadata access path, putting data on disk and metadata in byte-addressable NVM at runtime. Thus, data is accessed in a block from I/O the bus and metadata is accessed in a byte-addressable manner from the memory bus. Metadata access is significantly accelerated and metadata I/O is eliminated because metadata in NVM is no longer flushed back to the disk periodically. A lightweight consistency mechanism combining fine-grained versioning and transaction is introduced in the FSMAC. The FSMAC is implemented on a real NVDIMM platform and intensively evaluated under different workloads. Evaluation results show that the FSMAC accelerates the file system up to 49.2 times for synchronized I/O and 7.22 times for asynchronized I/O. Moreover, it can achieve significant performance speedup in network storage and database environment, especially for metadata-intensive or write-dominated workloads.

References

  1. N. Agrawal, W. J. Bolosky, J. R. Douceur, and J. R. Lorch. 2007. A five-year study of file-system metadata. In Proceedings of USENIX Conference on File and Storage Technologies (FAST’07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. AgigaTech. 2012. AGIGARAM DDR3 Non-Volatile DIMM. Retrieved July 3, 2015 from http://www.agigatech.com/ddr3.php.Google ScholarGoogle Scholar
  3. S. Best. 2002. JFS for Linux. http://www.free-soft.org/FSM/english/issue03/sbest.pdf.Google ScholarGoogle Scholar
  4. J. Bonwick and B. Moore. 2007. ZFS: The last word in file systems. http://pages.cs.wisc.edu/∼cs537-3/notes/Zhang_lecture/1_zfs_overview.pdf.Google ScholarGoogle Scholar
  5. G. W. Burr, B. N. Kurdi, J. C. Scott, C. H. Lam, K. Gopalakrishnan, and R. S. Shenoy. 2008. Overview of candidate device technologies for storage-class memory. IBM Journal of Research and Development 52, 4.5, 449--464. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. R. Card, T. Ts’o, and S. Tweedie. 1994. Design and implementation of the second extended file system. In Proceedings of the First Dutch International Symposium on Linux.Google ScholarGoogle Scholar
  7. A. M. Caulfield, T. I. Mollov, L. A. Eisner, A. De, J. Coburn, and S. Swanson. 2012. Providing safe, user space access to fast, solid state disks. In Proceedings of International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. J. X. Chen, Q. S. Wei, C. Chen, and L. K. Wu. 2013. FSMAC: A file system metadata accelerator with non-volatile memory. In Proceedings of the 29th IEEE Conference on Massive Data Storage (MSST’13).Google ScholarGoogle Scholar
  9. V. Chidambaram, T. Sharma, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. 2012. Consistency without ordering. In Proceedings of USENIX Conference on File and Storage Technologies (FAST’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. J. Coburn, A. M. Caulfield, A. Akel, L. M. Grupp, R. K. Gupta, R. Jhala, and S. Swanson. 2011. NV-Heaps: Making persistent objects fast and safe with next-generation, non-volatile memories. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. J. Condit, E. B. Nightingale, C. Frost, E. Ipek, B. Lee, D. Burger, and D. Coetzee. 2009. Better I/O through byte-addressable, persistent memory. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP’09). Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. I. H. Doh, J. Choi, D. Lee, and S. H. Noh. 2007. Exploiting non-volatile RAM to enhance flash file system performance. In Proceedings of the International Conference on Embedded Software (EMSOFT’07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. S. R. Dulloor, S. Kumar, A. Keshavamurthy, P. Lantz, D. Reddy, R. Sankaran, and J. Jackson. 2014. System software for persistent memory. In Proceedings of the 9th European Conference on Computer Systems (EuroSys’14). Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. D. Fryer, K. Sun, R. Mahmood, T. Cheng, S. Benjamin, A. Goel, and A. D. Brown. 2012. Recon: Verifying file system consistency at runtime. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. K. M. Greenan and E. L. Miller. 2006. Reliability mechanisms for file systems using non-volatile memory as a metadata store. In Proceedings of the International Conference on Embedded Software (EMSOFT’06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. L. M. Grupp, J. D. Davis, and S. Swanson. 2012. The bleak future of nand flash memory. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. J. E. Jang, E. Lee, K. Koh, and H. Bahn. 2012. Pairwise-FS: A novel file system structure for phase change memory. In Proceedings of the 18th IEEE Real-Time and Embedded Technology and Applications Symposium Work-in-Progress (WiP). 5--8.Google ScholarGoogle Scholar
  18. W. K. Josephson, L. A. Bongo, and K. Li. 2010. DFS: A file system for virtualized flash storage. In Proceedings of the 8th USENIX Conference on File and Storage Technologies (FAST’10). 85--100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. J. Jung, Y. Won, E. Kim, H. Shin, and B. Jeon. 2010. FRASH: Exploiting storage class memory in hybrid file system for hierarchical storage. ACM Transactions on Storage 6, 1, 1--25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. T. Kawahara. 2011. Scalable spin-transfer torque ram technology for normally-off computing. IEEE Design and Test of Computers 28, 1, 52--63. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. B. C. Lee, E. Ipek, O. Mutlu, and D. Burger. 2009. Architecting phase change memory as a scalable DRAM alternative. In Proceedings of the 36th International Symposium on Computer Architecture (ISCA’09). 2--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. E. Lee, H. Bahn, and S. H. Noh. 2013. Unioning of the buffer cache and journaling layers with non-volatile memory. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’13). Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. E. Lee, S. Yoo, J. E. Jang, and H. Bahn. 2012. Shortcut-JFS: A write efficient journaling file system for phase change memory. In Proceedings of the 28th IEEE Conference on Massive Data Storage (MSST’12).Google ScholarGoogle Scholar
  24. A. W. Leung, S. Pasupathy, G. Goodson, and E. L. Miller. 2008. Measurement and analysis of large-scale network file system workloads. In Proceedings of the USENIX Annual Technical Conference (ATC’08). 213--226. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Y. Lu, J. Shu, and W. Zheng. 2013. Extending the lifetime of flash-based storage through reducing write amplification from file systems. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’13). Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. A. Mathur, M. Cao, S. Bhattacharya, A. Dilger, A. Tomas, and L. Vivier. 2007. The new Ext4 file system: current status and future plans. 2007 Linux Symposium, Volume Two. 21--33.Google ScholarGoogle Scholar
  27. M. K. Mckusick, W. N. Joy, S. J. Leffler, and R. S. Fabry. 1986. Fsck—the UNIX file system check program. UNIX System Manager's Manual—4.3 BSD Virtual VAX-11 Version.Google ScholarGoogle Scholar
  28. I. Moraru, D. G. Andersen, M. Kaminsky, N. Binkert, N. Tolia, R. Munz, and P. Ranganathan. 2011. Persistent, Protected and Cached: Building Blocks for Main Memory Data Stores. Technique report of CMU-PDL-11-14.Google ScholarGoogle Scholar
  29. R. Nagar. 1997. Windows NT file system internals: A Developer's Guide. O’Reilly & Associates, Inc., Sebastopol, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. D. Narayanan and O. Hodson. 2012. Whole-system persistence. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Y. W. Park, S. H. Lim, C. Lee, and K. H. Park. 2008. PFFS: A scalable flash memory file system for the hybrid architecture of phase-change RAM and NAND flash. In Proceedings of the 2008 ACM Symposium on Applied Computing (SA’08). Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Y. W. Park and K. H. Park. 2011. High-performance scalable flash file system using virtual metadata storage with phase-change RAM. IEEE Transactions on Computers 60, 3, 321--334. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. M. K. Qureshi, V. Srinivasan, and J. A. Rivers. 2009. Scalable high performance main memory system using phase-change memory technology. In Proceedings of the 36th International Symposium on Computer Architecture (ISCA’09). 24--33. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. S. Raoux, G. W. Burr, M. J. Breitwisch, C. T. Rettner, Y.-C. Chen, R. M. Shelby, M. Salinga, D. Krebs, S.-H. Chen, H.-L. Lung, and C. H. Lam. 2008. Phase-change random access memory: A scalable technology. IBM Journal of Research and Development 52, 4/5, 465--479. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Hans Reiser. 2004. ReiserFS. Retrieved July 4, 2015 from www.namesys.com.Google ScholarGoogle Scholar
  36. Sourceforge. 2007. Fsmark. Retrieved July 4, 2015 from http://sourceforge.net/projects/fsmark/.Google ScholarGoogle Scholar
  37. A. Sweeney, D. Doucette, W. Hu, C. Anderson, M. Nishimoto, and G. Peck. 1996. Scalability in the XFS file system. In Proceedings of the USENIX Annual Technical Conference (ATC’96). Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. S. C. Tweedie. 1998. Journaling the Linux Ext2fs file system. In Proceedings of the 4th Annual Linux Expo, Durham, North Carolina.Google ScholarGoogle Scholar
  39. A. Traeger, E. Zadok, N. Joukov, and C. P. Wright. 2008. A nine year study of file system and storage benchmarking. ACM Transactions on Storage 4, 2, Article 5, 1--56. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. S. Venkataraman, N. Tolia, P. Ranganathan, and R. H. Campbell. 2011. Consistent and durable data structures for non-volatile byte-addressable memory. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Viking Technology. 2012. ArxCis-NV(TM) Non-Volatile DIMM. Retrieved July 4, 2015 from http://www.vikingtechnology.com/nvdimm-technology.Google ScholarGoogle Scholar
  42. H. Volos, S. Nalli, S. Panneerselvam, V. Varadarajan, P. Saxena, and M. M. Swift, 2014. Aerie: Flexible file-system interfaces to storage-class memory. In Proceedings of the 9th European Conference on Computer Systems (EuroSys’14). Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. H. Volos, A. J. Tack, and M. M. Swift. 2011. Mnemosyne: Lightweight persistent memory. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. A.-I. A. Wang, G. Kuenning, P. Reiher, and G. Popek. 2006. The conquest file system: Better performance through a disk/persistent-RAM hybrid design. ACM Transactions on Storage 2, 3, 309--348. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Wikipedia. 2009. Retrieved July 4, 2015 from Btrfs. en.wikipedia.org/wiki/Btrfs.Google ScholarGoogle Scholar
  46. X. Wu and A. L. N. Reddy. 2011. SCMFS: A file system for storage class memory. In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC’11). Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Accelerating File System Metadata Access with Byte-Addressable Nonvolatile Memory

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            • Published in

              cover image ACM Transactions on Storage
              ACM Transactions on Storage  Volume 11, Issue 3
              July 2015
              117 pages
              ISSN:1553-3077
              EISSN:1553-3093
              DOI:10.1145/2809503
              Issue’s Table of Contents

              Copyright © 2015 ACM

              Publisher

              Association for Computing Machinery

              New York, NY, United States

              Publication History

              • Published: 24 July 2015
              • Accepted: 1 April 2015
              • Revised: 1 October 2014
              • Received: 1 August 2013
              Published in tos Volume 11, Issue 3

              Permissions

              Request permissions about this article.

              Request Permissions

              Check for updates

              Qualifiers

              • research-article
              • Research
              • Refereed

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!