skip to main content
research-article
Public Access

An Analysis of Persistent Memory Use with WHISPER

Published:04 April 2017Publication History
Skip Abstract Section

Abstract

Emerging non-volatile memory (NVM) technologies promise durability with read and write latencies comparable to volatile memory (DRAM). We define Persistent Memory (PM) as NVM accessed with byte addressability at low latency via normal memory instructions. Persistent-memory applications ensure the consistency of persistent data by inserting ordering points between writes to PM allowing the construction of higher-level transaction mechanisms. An epoch is a set of writes to PM between ordering points.

To put systems research in PM on a firmer footing, we developed and analyzed a PM benchmark suite called WHISPER (Wisconsin-HP Labs Suite for Persistence) that comprises ten PM applications we gathered to cover all current interfaces to PM. A quantitative analysis reveals several insights: (a) only 4% of writes in PM-aware applications are to PM and the rest are to volatile memory, (b) software transactions are often implemented with 5 to 50 ordering points (c) 75% of epochs update exactly one 64B cache line, (d) 80% of epochs from the same thread depend on previous epochs from the same thread, while few epochs depend on epochs from other threads.

Based on our analysis, we propose the Hands-off Persistence System (HOPS) to track updates to PM in hardware. Current hardware design requires applications to force data to PM as each epoch ends. HOPS provides high-level ISA primitives for applications to express durability and ordering constraints separately and enforces them automatically, while achieving 24.3% better performance over current approaches to persistence.

References

  1. Crit-bit tree. cr.yp.to/critbit.html.Google ScholarGoogle Scholar
  2. Exim Internet Mailer. exim.org.Google ScholarGoogle Scholar
  3. In-kernel memory-mapped I/O tracing. kernel.org/doc/Documentation/trace/mmiotrace.txt.Google ScholarGoogle Scholar
  4. MySQL : The world's most popular open source database. mysql.com.Google ScholarGoogle Scholar
  5. pmem.io: Persistent memory programming blog. pmem.io.Google ScholarGoogle Scholar
  6. Redis, enhanced to use NVML's libpmemlog. github.com/pmem/redis.Google ScholarGoogle Scholar
  7. Secrets of the Ftrace function tracer. lwn.net/Articles/370423/.Google ScholarGoogle Scholar
  8. SysBench: a system performance benchmark. imysql.com/wp-content/uploads/2014/10/sysbench-manual.pdf.Google ScholarGoogle Scholar
  9. Arulraj, Joy and Pavlo, Andrew and Dulloor, Subramanya R. Let's Talk About Storage and Recovery Methods for Non-Volatile Memory Database Systems. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 707--722, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Bailey, Katelin A. and Hornyack, Peter and Ceze, Luis and Gribble, Steven D. and Levy, Henry M. Exploring Storage Class Memory with Key Value Stores. In Proceedings of the 1st Workshop on Interactions of NVM/FLASH with Operating Systems and Workloads, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Bhandari, Kumud and Chakrabarti, Dhruva R. and Boehm, Hans-J. Makalu: Fast Recoverable Allocation of Non-volatile Memory. In Proceedings of the ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, pages 677--694, 2016.Google ScholarGoogle Scholar
  12. Binkert, Nathan and Beckmann, Bradford and Black, Gabriel and Reinhardt, Steven K. and Saidi, Ali and Basu, Arkaprava and Hestness, Joel and Hower, Derek R. and Krishna, Tushar and Sardashti, Somayeh and Sen, Rathijit and Sewell, Korey and Shoaib, Muhammad and Vaish, Nilay and Hill, Mark D. and Wood, David A. The Gem5 Simulator. ACM SIGARCH Computer Architecture News, 39(2):1--7, August 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Cao Minh, Chi. Designing an Effective Hybrid Transactional Memory System. PhD thesis, Stanford University, Stanford, CA, USA, 2008.Google ScholarGoogle Scholar
  14. Carlson, Josiah L. Redis in Action. Manning Publications Co., Greenwich, CT, 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Chidambaram, Vijay and Pillai, Thanumalayan Sankaranarayana and Arpaci-Dusseau, Andrea C. and Arpaci-Dusseau, Remzi H. Optimistic Crash Consistency. In Proceedings of the 24th ACM Symposium on Operating Systems Principles, pages 228--243, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Christopher Frost. Improving File System Consistency and Durability with Patches and BPFS. PhD thesis, University of California, Los Angeles, 2010.Google ScholarGoogle Scholar
  17. Coburn, Joel and Caulfield, Adrian M. and Akel, Ameen and Grupp, Laura M. and Gupta, Rajesh K. and Jhala, Ranjit and Swanson, Steven. NV-Heaps: Making Persistent Objects Fast and Safe with Next-generation, Non-volatile Memories. In Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems, pages 105--118, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Condit, Jeremy and Nightingale, Edmund B. and Frost, Christopher and Ipek, Engin and Lee, Benjamin and Burger, Doug and Coetzee, Derrick. Better I/O Through Byte-addressable, Persistent Memory. In Proceedings of the 22nd ACM SIGOPS Symposium on Operating Systems Principles, pages 133--146, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Intel Corporation. Intel architecture instruction set extensions programming reference, August 2015.Google ScholarGoogle Scholar
  20. D. Williams. Replace pcommit with ADR or directed flushing. lwn.net/Articles/694134/.Google ScholarGoogle Scholar
  21. Dulloor, Subramanya R. and Kumar, Sanjay and Keshavamurthy, Anil and Lantz, Philip and Reddy, Dheeraj and Sankaran, Rajesh and Jackson, Jeff. System Software for Persistent Memory. In Proceedings of the 9th European Conference on Computer Systems, pages 15:1--15:15, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Dulloor, Subramanya R. and Roy, Amitabha and Zhao, Zheguang and Sundaram, Narayanan and Satish, Nadathur and Sankaran, Rajesh and Jackson, Jeff and Schwan, Karsten. Data Tiering in Heterogeneous Memory Systems. In Proceedings of the 11th European Conference on Computer Systems, pages 15:1--15:16, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Hewlett Packard Enterprise. Persistent memory. hpe.com/us/en/servers/persistent-memory.html.Google ScholarGoogle Scholar
  24. Fitzpatrick, Brad. Distributed Caching with Memcached. Linux Journal, 2004(124):5--, August 2004.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Intel Newsroom. Intel and Micron produce breakthrough memory technology. newsroom.intel.com/community/intel_newsroom /blog/2015/07/28/intel-and-micron-produce-breakthrough-memory-technology, July 2015.Google ScholarGoogle Scholar
  26. Joshi, Arpit and Nagarajan, Vijay and Cintra, Marcelo and Viglas, Stratis. Efficient Persist Barriers for Multicores. In Proceedings of the 48th International Symposium on Microarchitecture, pages 660--671, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Kallman, Robert and Kimura, Hideaki and Natkins, Jonathan and Pavlo, Andrew and Rasin, Alexander and Zdonik, Stanley and Jones, Evan P. C. and Madden, Samuel and Stonebraker, Michael and Zhang, Yang and Hugg, John and Abadi, Daniel J. H-store: A High-performance, Distributed Main Memory Transaction Processing System. Proceedings of the VLDB Endowment, 1(2):1496--1499, August 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Kolli, Aasheesh and Pelley, Steven and Saidi, Ali and Chen, Peter M. and Wenisch, Thomas F. High-Performance Transactions for Persistent Memories. In Proceedings of the 21st International Conference on Architectural Support for Programming Languages and Operating Systems, pages 399--411, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Kolli, Aasheesh and Rosen, Jeff and Diestelhorst, Stephan and Saidi, Ali and Pelley, Steven and Liu, Sihang and Chen, Peter M. and Wenisch, Thomas F. Delegated Persist Ordering. In Proceedings of the 49th International Symposium on Microarchitecture, pages 1--13, 2016. Google ScholarGoogle ScholarCross RefCross Ref
  30. Liu, Ren-Shuo and Shen, De-Yu and Yang, Chia-Lin and Yu, Shun-Chih and Wang, Cheng-Yuan Michael. NVM Duet: Unified Working Memory and Persistent Store Architecture. In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems, pages 455--470, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Lu, Youyou and Shu, Jiwu and Sun, Long. Blurred Persistence: Efficient Transactions in Persistent Memory. ACM Transactions on Storage, 12(1):3:1--3:29, January 2016.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Lu, Youyou and Shu, Jiwu and Sun, Long and Mutlu, Onur. Loose-ordering consistency for persistent memory. In International Conference on Computer Design, pages 216--223. IEEE, 2014. Google ScholarGoogle ScholarCross RefCross Ref
  33. Minh, Chi Cao and Chung, JaeWoong and Kozyrakis, Christos and Olukotun, Kunle. STAMP: Stanford transactional applications for multi-processing. In International Symposium on Workload Characterization, pages 35--46. IEEE, 2008.Google ScholarGoogle Scholar
  34. Narayanan, Dushyanth and Hodson, Orion. Whole-system Persistence. In Proceedings of the 17th International Conference on Architectural Support for Programming Languages and Operating Systems, pages 401--410, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Pelley, Steven and Chen, Peter M. and Wenisch, Thomas F. Memory Persistency. In Proceeding of the 41st Annual International Symposium on Computer Architecuture, pages 265--276, 2014. Google ScholarGoogle ScholarCross RefCross Ref
  36. Ren, Jinglei and Zhao, Jishen and Khan, Samira and Choi, Jongmoo and Wu, Yongwei and Mutlu, Onur. ThyNVM: Enabling Software-transparent Crash Consistency in Persistent Memory Systems. In Proceedings of the 48th International Symposium on Microarchitecture, pages 672--685, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. SNIA. NVM Programming Technical Work Group. snia.org/forums/sssi/nvmp.Google ScholarGoogle Scholar
  38. V. Tarasov, E. Zadok, and S. Shepler. Filebench: A flexible framework for file system benchmarking. ;login: The USENIX Magazine, 41(1):6--12, March 2016.Google ScholarGoogle Scholar
  39. Volos, Haris and Tack, Andres Jaan and Swift, Michael M. Mnemosyne: Lightweight Persistent Memory. In Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems, pages 91--104, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Wu, Xiaojian and Reddy, A. L. Narasimha. SCMFS: A File System for Storage Class Memory. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 39:1--39:11, 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Xu, Jian and Swanson, Steven. NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories. In Proceedings of the 14th Usenix Conference on File and Storage Technologies, pages 323--338, 2016.Google ScholarGoogle Scholar
  42. Yen, Luke and Bobba, Jayaram and Marty, Michael R. and Moore, Kevin E. and Volos, Haris and Hill, Mark D. and Swift, Michael M. and Wood, David A. LogTM-SE: Decoupling Hardware Transactional Memory from Caches. In Proceedings of the 13th IEEE International Symposium on High Performance Computer Architecture, pages 261--272, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Zhao, Jishen and Li, Sheng and Yoon, Doe Hyun and Xie, Yuan and Jouppi, Norman P. Kiln: Closing the Performance Gap Between Systems with and Without Persistence Support. In Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture, pages 421--432, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. An Analysis of Persistent Memory Use with WHISPER

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!