skip to main content
research-article

HEAPO: Heap-Based Persistent Object Store

Published:29 December 2014Publication History
Skip Abstract Section

Abstract

In this work, we developed a Heap-Based Persistent Object Store (HEAPO) to manage persistent objects in byte-addressable Nonvolatile RAM (NVRAM). HEAPO defines its own persistent heap layout, the persistent object format, name space organization, object sharing and protection mechanism, and undo-only log-based crash recovery, all of which are effectively tailored for NVRAM. We put our effort into developing a lightweight and flexible layer to exploit the DRAM-like access latency of NVRAM. To address this objective, we developed (i) a native management layer for NVRAM to eliminate redundancy between in-core and on-disk copies of the metadata, (ii) an expandable object format, (iii) a burst trie-based global name space with local name space caching, (iv) static address binding, and (v) minimal logging for undo-only crash recovery. We implemented HEAPO at commodity OS (Linux 2.6.32) and measured the performance. By eliminating metadata redundancy, HEAPO improved the speed of creating, attaching, and expanding an object by 1.3×, 4.5×, and 3.8×, respectively, compared to memory-mapped file-based persistent object store. Burst trie-based name space organization of HEAPO yielded 7.6× better lookup performance compared to hashed B-tree-based name space of EXT4. We modified memcachedb to use HEAPO in maintaining its search structure. For hash table update, HEAPO-based memcachedb yielded 3.4× performance improvement against original memcachedb implementation which uses mmap() over ramdisk approach to maintain the key-value store in memory.

References

  1. A. Akel, A. M. Caulfield, T. I. Mollov, R. K. Gupta, and S. Swanson. 2011. Onyx: A prototype phase change memory storage array. In Proc. Workshop on Hot Topics in Storage and File Systems (HotStorage’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. S. Ames, N. Bobb, K. M. Greenan, O. S. Hofmann, M. W. Storer, C. Maltzahn, E. L. Miller, and S. A. Brandt. 2006. LiFS: An attribute-rich file system for storage class memories. In Proc. of IEEE Conference on Mass Storage Systems and Technologies (MSST’06).Google ScholarGoogle Scholar
  3. Apache. 2009. Cassandra. Retrieved from http://cassandra.apache.org/.Google ScholarGoogle Scholar
  4. Anirudh Badam and Vivek S. Pai. 2011. SSDAlloc: Hybrid SSD/RAM memory management made easy. In Proc. of USENIX Conference on Networked Systems Design and Implementation (NSDI’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. M. Cao, T. Y. Tso, B. Pulavarty, S. Bhattacharya, A. Dilger, and A. Tomas. 2005. State of the art: Where we are with the ext3 filesystem. In Proc. of the Ottawa Linux Symposium (OLS’05).Google ScholarGoogle Scholar
  6. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber. 2008. Bigtable: A distributed storage system for structured data. ACM Trans. Comput. Syst. (TOCS) 26, 2 (2008). Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Kristina Chodorow. 2010. Introduction to MongoDB. In Free and Open Source Software Developers European Meeting (FOSDEM’10).Google ScholarGoogle Scholar
  8. S. Chu. 2008. Memcachedb. Retrieved from http://memcachedb.org.Google ScholarGoogle Scholar
  9. J. Coburn, A. M. Caulfield, A. Akel, L. M. Grupp, R. K. Gupta, R. Jhala, and S. Swanson. 2011. Nv-heaps: Making persistent objects fast and safe with next-generation, non-volatile memories. In Proc. of International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. J. Condit, E. B. Nightingale, C. Frost, E. Ipek, B. Lee, D. Burger, and D. Coetzee. 2009. Better I/O through byte-addressable, persistent memory. In Proc. of the ACM SIGOPS Symposium on Operating Systems Principles (SOSP’09). Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. A. Dearle, J. Rosenberg, F. Henskens, F. Vaughan, and K. Maciunas. 1992. An examination of operating system support for persistent object systems. In Proc. of Hawaii International Conference on System Sciences (HICSS’02).Google ScholarGoogle Scholar
  12. Diblotechnology. 2013. Memory Channel Storage. Retrieved from http://www.diablo-technologies.com/products/mcs.html.Google ScholarGoogle Scholar
  13. R. F. Freitas and W. W. Wilcke. 2008. Storage-class memory: The next storage system technology. IBM J. Res. Dev. 52, 4.5 (2008), 439--447. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. J. Guerra, L. Mármol, D. Campello, C. Crespo, R. Rangaswami, and J. Wei. 2012. Software persistent memory. In Proc. of USENIX Annual Technical Conference (ATC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. T. Harter, C. Dragga, M. Vaughn, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. 2012. A file is not a file: Understanding the I/O behavior of Apple desktop applications. ACM Trans. Comput. Syst. (TOCS) 30, 3 (2012). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. S. Heinz, J. Zobel, and H. E. Williams. 2002. Burst tries: A fast, efficient data structure for string keys. ACM Trans. Inf. Syst. (TOIS) 20, 2 (2002), 192--223. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. I. Jantunen, J. Hämäläinen, T. Korhonen, H. Kaaja, J. Jantunen, and S. Boldyrev. 2010. System architecture for mobile-phone-readable RF memory tags. In Proc. of UBICOMM.Google ScholarGoogle Scholar
  18. J. Jung, Y. Won, E. Kim, H. Shin, and B. Jeon. 2010. FRASH: Exploiting storage class memory in hybrid file system for hierarchical storage. ACM Trans. Storage (TOS) 6, 1 (2010). Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Y. M. Kang, H. J. Joo, J. H. Park, S. K. Kang, J. H. Kim, S. G. Oh, H. S. Kim, Y. J. Kang, J. Y. Jung, D. Y. Choi, and others. 2006. World smallest 0.34/spl mu/m∼ COB cell 1T1C 64Mb FRAM with new sensing architecture and highly reliable MOCVD PZT integration technology. In Proc. of Symposium on VLSI Technology.Google ScholarGoogle Scholar
  20. Y. Kang, J. Yang, and E. L. Miller. 2011. Object-based SCM: An efficient interface for storage class memories. In Proc. of IEEE Conference on Mass Storage Systems and Technologies (MSST’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. S. Kim and C. H. Lam. 2012. Transition of memory technologies. In Proc. 2012 International Symposium on VLSI Technology, Systems, and Applications (VLSI-TSA), IEEE.Google ScholarGoogle Scholar
  22. M. H. Kryder and C. S. Kim. 2009. After hard drives? What comes next? IEEE Trans. Magn. 45, 10 (2009), 3406--3413.Google ScholarGoogle ScholarCross RefCross Ref
  23. C. Lever and Sun-Netscape Alliance. 2000. Linux Kernel Hash Table Behavior: Analysis and Improvements. Technical Report 00-1. Center for Information Technology Integration, University of Michigan, Ann Arbor, MI (2000).Google ScholarGoogle Scholar
  24. Z. Liu, B. Wang, P. Carpenter, D. Li, J. S. Vetter, and W. Yu. 2012. PCM-based durable write cache for fast disk I/O. In Proc. of International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS’’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. A. Mathur, M. Cao, S. Bhattacharya, A. Dilger, A. Tomas, and L. Vivier. 2007. The new ext4 file system: Current status and future plans. In Proc. of Ottawa Linux Symposium (OLS’07).Google ScholarGoogle Scholar
  26. I. Moraru, D. G. Andersen, M. Kaminsky, N. Binkert, N. Tolia, R. Munz, and P. Ranganathan. 2011. Persistent, protected and cached: Building blocks for main memory data stores. In Proc. of Annual Parallel Data Lab Workshop and Retreat.Google ScholarGoogle Scholar
  27. J. E. B. Moss. 1990. Garbage collecting persistent object stores. In Proc. Workshop on Garbage Collection (GC’90).Google ScholarGoogle Scholar
  28. Microsoft Corp. msdn. 2010. Ramdisk. Retrieved from msdn.microsoft.com/en-us/library/ff544551(VS.85). aspx.Google ScholarGoogle Scholar
  29. D. Narayanan and O. Hodson. 2012. Whole-system persistence. In Proc. of International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. A. Olson, K. Bostic, and M. Seltzer. 1999. Berkeley DB. In Proc. of the FREENIX Track: 1999 USENIX Annual Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. T. Perez. 2012. Evaluation of System-Level Impacts of A Persistent Main Memory Architecture. Ph.D. Dissertation. Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, RS, Brazil.Google ScholarGoogle Scholar
  32. M. K. Qureshi, V. Srinivasan, and J. A. Rivers. 2009. Scalable high performance main memory system using phase-change memory technology. In ACM SIGARCH Comput. Archit. News, 37, 3 (June 2009). 24--33. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Objective Analysis Semiconductor Market Research. 2009. PCM Becomes a Reality: Memory Design will Never be the Same. Retrieved from http://www.objective-analysis.com/uploads/2009-08-03_Objective_ Analysis_PCM_White_Paper.pdf.Google ScholarGoogle Scholar
  34. E. Shekita and M. Zwilling. 1990. Cricket: A mapped, persistent object store. In Proc. Workshop on the Persistent Object Systems.Google ScholarGoogle Scholar
  35. Sun Microsystems Inc. 2004. Solaris ZFS file storage solution. Solaris 10 Data Sheets.Google ScholarGoogle Scholar
  36. A. Sweeney, D. Doucette, W. Hu, C. Anderson, M. Nishimoto, and G. Peck. 1996. Scalability in the XFS file system. In Proc. of the USENIX 1996 Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. S. Venkataraman, N. Tolia, P. Ranganathan, and R. H. Campbell. 2011. Consistent and durable data structures for non-volatile byte-addressable memory. In Proc. of USENIX Conference on File and Storage Technologies (FAST’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Vikingtechnology. 2013. NV-DIMM. Retrieved from http://www.vikingtechnology.com/nvdimm-technology.Google ScholarGoogle Scholar
  39. H. Volos and M. Swift. 2011. Storage systems for storage-class memory. In Proc. of Annual Non-Volatile Memories Workshop (NVMW’11).Google ScholarGoogle Scholar
  40. H. Volos, A. J. Tack, and M. M. Swift. 2011. Mnemosyne: Lightweight persistent memory. In Proc. of International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. A. I. A. Wang, G. Kuenning, P. Reiher, and G. Popek. 2006. The conquest file system: Better performance through a disk/persistent-RAM hybrid design. ACM Trans. Storage (TOS) 2, 3 (Aug. 2006), 309--348. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. C. Wang, S. S. Vazhkudai, X. Ma, F. Meng, Y. Kim, and C. Engelmann. 2012. NVMalloc: Exposing an aggregate SSD store as a memory partition in extreme-scale machines. In Proc. of International Parallel and Distributed Processing Symposium (IPDPS’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. K. L. Wang and P. Khalili Amiri. 2012. Nonvolatile spintronics: Perspectives on instant-on nonvolatile nanoelectronic systems. SPIN, 2, 2 (June 2012).Google ScholarGoogle ScholarCross RefCross Ref
  44. P. W. Y. Wong, B. Pulavarty, S. Nagar, J. Morgan, J. Lahr, B. Hartner, H. Franke, and S. Bhattacharya. 2002. Improving linux block I/O for enterprise workloads. In Proc. of Ottawa Linux Symposium.Google ScholarGoogle Scholar
  45. X. Wu and A. L. N. Reddy. 2011. SCMFS: A file system for storage class memory. In Proc. of International Conference for High Performance Computing, Networking, Storage and Analysis (SC’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Jung H. Yoon, Hillery C. Hunter, and Gary A. Tressler. 2013. Flash & DRAM Si scaling challenges, emerging non-volatile memory technology enablement——Implications to enterprise storage and server compute systems. In Flash Memory Summit.Google ScholarGoogle Scholar

Index Terms

  1. HEAPO: Heap-Based Persistent Object Store

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Storage
          ACM Transactions on Storage  Volume 11, Issue 1
          February 2015
          100 pages
          ISSN:1553-3077
          EISSN:1553-3093
          DOI:10.1145/2705611
          • Editor:
          • Darrell Long
          Issue’s Table of Contents

          Copyright © 2014 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 29 December 2014
          • Accepted: 1 May 2014
          • Received: 1 February 2014
          Published in tos Volume 11, Issue 1

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!