Abstract

Emerging non-volatile memory (NVM) technologies promise durability with read and write latencies comparable to volatile memory (DRAM). We define Persistent Memory (PM) as NVM accessed with byte addressability at low latency via normal memory instructions. Persistent-memory applications ensure the consistency of persistent data by inserting ordering points between writes to PM allowing the construction of higher-level transaction mechanisms. An epoch is a set of writes to PM between ordering points.
To put systems research in PM on a firmer footing, we developed and analyzed a PM benchmark suite called WHISPER (Wisconsin-HP Labs Suite for Persistence) that comprises ten PM applications we gathered to cover all current interfaces to PM. A quantitative analysis reveals several insights: (a) only 4% of writes in PM-aware applications are to PM and the rest are to volatile memory, (b) software transactions are often implemented with 5 to 50 ordering points (c) 75% of epochs update exactly one 64B cache line, (d) 80% of epochs from the same thread depend on previous epochs from the same thread, while few epochs depend on epochs from other threads.
Based on our analysis, we propose the Hands-off Persistence System (HOPS) to track updates to PM in hardware. Current hardware design requires applications to force data to PM as each epoch ends. HOPS provides high-level ISA primitives for applications to express durability and ordering constraints separately and enforces them automatically, while achieving 24.3% better performance over current approaches to persistence.
- Crit-bit tree. cr.yp.to/critbit.html.Google Scholar
- Exim Internet Mailer. exim.org.Google Scholar
- In-kernel memory-mapped I/O tracing. kernel.org/doc/Documentation/trace/mmiotrace.txt.Google Scholar
- MySQL : The world's most popular open source database. mysql.com.Google Scholar
- pmem.io: Persistent memory programming blog. pmem.io.Google Scholar
- Redis, enhanced to use NVML's libpmemlog. github.com/pmem/redis.Google Scholar
- Secrets of the Ftrace function tracer. lwn.net/Articles/370423/.Google Scholar
- SysBench: a system performance benchmark. imysql.com/wp-content/uploads/2014/10/sysbench-manual.pdf.Google Scholar
- Arulraj, Joy and Pavlo, Andrew and Dulloor, Subramanya R. Let's Talk About Storage and Recovery Methods for Non-Volatile Memory Database Systems. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 707--722, 2015. Google Scholar
Digital Library
- Bailey, Katelin A. and Hornyack, Peter and Ceze, Luis and Gribble, Steven D. and Levy, Henry M. Exploring Storage Class Memory with Key Value Stores. In Proceedings of the 1st Workshop on Interactions of NVM/FLASH with Operating Systems and Workloads, 2013. Google Scholar
Digital Library
- Bhandari, Kumud and Chakrabarti, Dhruva R. and Boehm, Hans-J. Makalu: Fast Recoverable Allocation of Non-volatile Memory. In Proceedings of the ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, pages 677--694, 2016.Google Scholar
- Binkert, Nathan and Beckmann, Bradford and Black, Gabriel and Reinhardt, Steven K. and Saidi, Ali and Basu, Arkaprava and Hestness, Joel and Hower, Derek R. and Krishna, Tushar and Sardashti, Somayeh and Sen, Rathijit and Sewell, Korey and Shoaib, Muhammad and Vaish, Nilay and Hill, Mark D. and Wood, David A. The Gem5 Simulator. ACM SIGARCH Computer Architecture News, 39(2):1--7, August 2011. Google Scholar
Digital Library
- Cao Minh, Chi. Designing an Effective Hybrid Transactional Memory System. PhD thesis, Stanford University, Stanford, CA, USA, 2008.Google Scholar
- Carlson, Josiah L. Redis in Action. Manning Publications Co., Greenwich, CT, 2013.Google Scholar
Digital Library
- Chidambaram, Vijay and Pillai, Thanumalayan Sankaranarayana and Arpaci-Dusseau, Andrea C. and Arpaci-Dusseau, Remzi H. Optimistic Crash Consistency. In Proceedings of the 24th ACM Symposium on Operating Systems Principles, pages 228--243, 2013. Google Scholar
Digital Library
- Christopher Frost. Improving File System Consistency and Durability with Patches and BPFS. PhD thesis, University of California, Los Angeles, 2010.Google Scholar
- Coburn, Joel and Caulfield, Adrian M. and Akel, Ameen and Grupp, Laura M. and Gupta, Rajesh K. and Jhala, Ranjit and Swanson, Steven. NV-Heaps: Making Persistent Objects Fast and Safe with Next-generation, Non-volatile Memories. In Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems, pages 105--118, 2011. Google Scholar
Digital Library
- Condit, Jeremy and Nightingale, Edmund B. and Frost, Christopher and Ipek, Engin and Lee, Benjamin and Burger, Doug and Coetzee, Derrick. Better I/O Through Byte-addressable, Persistent Memory. In Proceedings of the 22nd ACM SIGOPS Symposium on Operating Systems Principles, pages 133--146, 2009. Google Scholar
Digital Library
- Intel Corporation. Intel architecture instruction set extensions programming reference, August 2015.Google Scholar
- D. Williams. Replace pcommit with ADR or directed flushing. lwn.net/Articles/694134/.Google Scholar
- Dulloor, Subramanya R. and Kumar, Sanjay and Keshavamurthy, Anil and Lantz, Philip and Reddy, Dheeraj and Sankaran, Rajesh and Jackson, Jeff. System Software for Persistent Memory. In Proceedings of the 9th European Conference on Computer Systems, pages 15:1--15:15, 2014. Google Scholar
Digital Library
- Dulloor, Subramanya R. and Roy, Amitabha and Zhao, Zheguang and Sundaram, Narayanan and Satish, Nadathur and Sankaran, Rajesh and Jackson, Jeff and Schwan, Karsten. Data Tiering in Heterogeneous Memory Systems. In Proceedings of the 11th European Conference on Computer Systems, pages 15:1--15:16, 2016. Google Scholar
Digital Library
- Hewlett Packard Enterprise. Persistent memory. hpe.com/us/en/servers/persistent-memory.html.Google Scholar
- Fitzpatrick, Brad. Distributed Caching with Memcached. Linux Journal, 2004(124):5--, August 2004.Google Scholar
Digital Library
- Intel Newsroom. Intel and Micron produce breakthrough memory technology. newsroom.intel.com/community/intel_newsroom /blog/2015/07/28/intel-and-micron-produce-breakthrough-memory-technology, July 2015.Google Scholar
- Joshi, Arpit and Nagarajan, Vijay and Cintra, Marcelo and Viglas, Stratis. Efficient Persist Barriers for Multicores. In Proceedings of the 48th International Symposium on Microarchitecture, pages 660--671, 2015. Google Scholar
Digital Library
- Kallman, Robert and Kimura, Hideaki and Natkins, Jonathan and Pavlo, Andrew and Rasin, Alexander and Zdonik, Stanley and Jones, Evan P. C. and Madden, Samuel and Stonebraker, Michael and Zhang, Yang and Hugg, John and Abadi, Daniel J. H-store: A High-performance, Distributed Main Memory Transaction Processing System. Proceedings of the VLDB Endowment, 1(2):1496--1499, August 2008. Google Scholar
Digital Library
- Kolli, Aasheesh and Pelley, Steven and Saidi, Ali and Chen, Peter M. and Wenisch, Thomas F. High-Performance Transactions for Persistent Memories. In Proceedings of the 21st International Conference on Architectural Support for Programming Languages and Operating Systems, pages 399--411, 2016. Google Scholar
Digital Library
- Kolli, Aasheesh and Rosen, Jeff and Diestelhorst, Stephan and Saidi, Ali and Pelley, Steven and Liu, Sihang and Chen, Peter M. and Wenisch, Thomas F. Delegated Persist Ordering. In Proceedings of the 49th International Symposium on Microarchitecture, pages 1--13, 2016. Google Scholar
Cross Ref
- Liu, Ren-Shuo and Shen, De-Yu and Yang, Chia-Lin and Yu, Shun-Chih and Wang, Cheng-Yuan Michael. NVM Duet: Unified Working Memory and Persistent Store Architecture. In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems, pages 455--470, 2014. Google Scholar
Digital Library
- Lu, Youyou and Shu, Jiwu and Sun, Long. Blurred Persistence: Efficient Transactions in Persistent Memory. ACM Transactions on Storage, 12(1):3:1--3:29, January 2016.Google Scholar
Digital Library
- Lu, Youyou and Shu, Jiwu and Sun, Long and Mutlu, Onur. Loose-ordering consistency for persistent memory. In International Conference on Computer Design, pages 216--223. IEEE, 2014. Google Scholar
Cross Ref
- Minh, Chi Cao and Chung, JaeWoong and Kozyrakis, Christos and Olukotun, Kunle. STAMP: Stanford transactional applications for multi-processing. In International Symposium on Workload Characterization, pages 35--46. IEEE, 2008.Google Scholar
- Narayanan, Dushyanth and Hodson, Orion. Whole-system Persistence. In Proceedings of the 17th International Conference on Architectural Support for Programming Languages and Operating Systems, pages 401--410, 2012. Google Scholar
Digital Library
- Pelley, Steven and Chen, Peter M. and Wenisch, Thomas F. Memory Persistency. In Proceeding of the 41st Annual International Symposium on Computer Architecuture, pages 265--276, 2014. Google Scholar
Cross Ref
- Ren, Jinglei and Zhao, Jishen and Khan, Samira and Choi, Jongmoo and Wu, Yongwei and Mutlu, Onur. ThyNVM: Enabling Software-transparent Crash Consistency in Persistent Memory Systems. In Proceedings of the 48th International Symposium on Microarchitecture, pages 672--685, 2015. Google Scholar
Digital Library
- SNIA. NVM Programming Technical Work Group. snia.org/forums/sssi/nvmp.Google Scholar
- V. Tarasov, E. Zadok, and S. Shepler. Filebench: A flexible framework for file system benchmarking. ;login: The USENIX Magazine, 41(1):6--12, March 2016.Google Scholar
- Volos, Haris and Tack, Andres Jaan and Swift, Michael M. Mnemosyne: Lightweight Persistent Memory. In Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems, pages 91--104, 2011. Google Scholar
Digital Library
- Wu, Xiaojian and Reddy, A. L. Narasimha. SCMFS: A File System for Storage Class Memory. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 39:1--39:11, 2011.Google Scholar
Digital Library
- Xu, Jian and Swanson, Steven. NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories. In Proceedings of the 14th Usenix Conference on File and Storage Technologies, pages 323--338, 2016.Google Scholar
- Yen, Luke and Bobba, Jayaram and Marty, Michael R. and Moore, Kevin E. and Volos, Haris and Hill, Mark D. and Swift, Michael M. and Wood, David A. LogTM-SE: Decoupling Hardware Transactional Memory from Caches. In Proceedings of the 13th IEEE International Symposium on High Performance Computer Architecture, pages 261--272, 2007. Google Scholar
Digital Library
- Zhao, Jishen and Li, Sheng and Yoon, Doe Hyun and Xie, Yuan and Jouppi, Norman P. Kiln: Closing the Performance Gap Between Systems with and Without Persistence Support. In Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture, pages 421--432, 2013. Google Scholar
Digital Library
Index Terms
An Analysis of Persistent Memory Use with WHISPER
Recommendations
An Analysis of Persistent Memory Use with WHISPER
ASPLOS '17: Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating SystemsEmerging non-volatile memory (NVM) technologies promise durability with read and write latencies comparable to volatile memory (DRAM). We define Persistent Memory (PM) as NVM accessed with byte addressability at low latency via normal memory ...
An Analysis of Persistent Memory Use with WHISPER
Asplos'17Emerging non-volatile memory (NVM) technologies promise durability with read and write latencies comparable to volatile memory (DRAM). We define Persistent Memory (PM) as NVM accessed with byte addressability at low latency via normal memory ...
NVM duet: unified working memory and persistent store architecture
ASPLOS '14: Proceedings of the 19th international conference on Architectural support for programming languages and operating systemsEmerging non-volatile memory (NVM) technologies have gained a lot of attention recently. The byte-addressability and high density of NVM enable computer architects to build large-scale main memory systems. NVM has also been shown to be a promising ...







Comments