Abstract
Non-volatile memory is expected to coexist with (or even displace) volatile DRAM for main memory in upcoming architectures. This has led to increasing interest in the problem of designing and specifying durable data structures that can recover from system crashes. Data structures may be designed to satisfy stricter or weaker durability guarantees to provide a balance between the strength of the provided guarantees and performance overhead. This paper proposes three novel implementations of a concurrent lock-free queue. These implementations illustrate algorithmic challenges in building persistent lock-free data structures with different levels of durability guarantees. In presenting these challenges, the proposed algorithmic designs, and the different durability guarantees, we hope to shed light on ways to build a wide variety of durable data structures. We implemented the various designs and compared their performance overhead to a simple queue design for standard (volatile) memory.
- Marcos Kawazoe Aguilera, Wei Chen, and Sam Toueg. 2000. Failure Detection and Consensus in the Crash-recovery Model. Distrib. Comput. 13, 2 (April 2000), 99--125. Google Scholar
Digital Library
- Marcos K Aguilera and Svend Frølund. 2003. Strict linearizability and the power of aborting. Technical Report HPL-2003-241 (2003), 25. http://www.hpl.hp.com/techreports/2003/HPL-2003-241.pdfGoogle Scholar
- Kumud Bhandari, Dhruva R. Chakrabarti, and Hans-J. Boehm. 2016. Makalu: Fast Recoverable Allocation of Non-volatile Memory. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2016). ACM, New York, NY, USA, 677--694. Google Scholar
Digital Library
- Dhruva R. Chakrabarti, Hans-J. Boehm, and Kumud Bhandari. 2014. Atlas: Leveraging Locks for Non-volatile Memory Consistency. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications (OOPSLA '14). ACM, New York, NY, USA, 433--452. Google Scholar
Digital Library
- Andreas Chatzistergiou, Marcelo Cintra, and Stratis D. Viglas. 2015. REWIND: Recovery Write-Ahead System for In-Memory Non-Volatile Data-Structures. PVLDB 8, 5 (2015), 497--508. http://www.vldb.org/pvldb/vol8/p497-chatzistergiou.pdf Google Scholar
Digital Library
- Ping Chi, Wang-Chien Lee, and Yuan Xie. 2014. Making B+-tree Efficient in PCM-based Main Memory. In Proceedings of the 2014 International Symposium on Low Power Electronics and Design (ISLPED '14). ACM, New York, NY, USA, 69--74. Google Scholar
Digital Library
- Joel Coburn, Adrian M. Caulfield, Ameen Akel, Laura M. Grupp, Rajesh K. Gupta, Ranjit Jhala, and Steven Swanson. 2011. NV-Heaps: Making Persistent Objects Fast and Safe with Next-generation, Non-volatile Memories. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XVI). ACM, New York, NY, USA, 105--118. Google Scholar
Digital Library
- Nachshon Cohen, Michal Friedman, and James R. Larus. 2017. Efficient Logging in Non-volatile Memory by Exploiting Coherency Protocols. Proc. ACM Program. Lang. 1, OOPSLA, Article 67 (Oct. 2017), 24 pages. Google Scholar
Digital Library
- Jeremy Condit, Edmund B. Nightingale, Christopher Frost, Engin Ipek, Benjamin Lee, Doug Burger, and Derrick Coetzee. 2009. Better I/O Through Byte-addressable, Persistent Memory. In Proceedings of the ACM SIGOPS 22Nd Symposium on Operating Systems Principles (SOSP '09). ACM, New York, NY, USA, 133--146. Google Scholar
Digital Library
- Fatcache 2013. twitter/fatcache: Memcache on SSD. https://github.com/twitter/fatcache. (2013).Google Scholar
- Michal Friedman, Maurice Herlihy, Virendra Marathe, and Erez Petrank. 2018. A Persistent Lock-Free Queue for Non-Volatile Memory. (2018). http://www.cs.technion.ac.il/~erez/Papers/nvm-queue-full.pdfGoogle Scholar
- Ellis Giles, Kshitij Doshi, and Peter J. Varman. 2015. SoftWrAP: A lightweight framework for transactional support of storage class memory. In 2015 31st Symposium on Mass Storage Systems and Technologies (MSST), Vol. 00. 1--14.Google Scholar
- Rachid Guerraoui and Ron R. Levy. 2004. Robust Emulations of Shared Memory in a Crash-Recovery Model. In in: Proceedings of the 24th IEEE International Conference on Distributed Computing Systems, ICDCS. 400--407. Google Scholar
Digital Library
- Maurice Herlihy. 1990. A Methodology for Implementing Highly Concurrent Data Structures. In Proceedings of the Second ACM SIGPLAN Symposium on Principles & Practice of Parallel Programming (PPOPP '90). ACM, New York, NY, USA, 197--206. Google Scholar
Digital Library
- Maurice P. Herlihy and Jeannette M. Wing. 1990. Linearizability: A Correctness Condition for Concurrent Objects. ACM Trans. Program. Lang. Syst. 12, 3 (July 1990), 463--492. Google Scholar
Digital Library
- Intel-Architecture-Manual 2017. Intel Architectures Software Developer Manual. https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf. (2017).Google Scholar
- Joseph Izraelevitz, Hammurabi Mendes, and Michael L. Scott. 2016. Linearizability of Persistent Memory Objects under a Full-System-Crash Failure Model. In DISC 2016, Paris, France, 2016.Google Scholar
- Aasheesh Kolli, Steven Pelley, Ali Saidi, Peter M. Chen, and Thomas F. Wenisch. 2016. High-Performance Transactions for Persistent Memories. In Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 16). ACM, New York, NY, USA, 399--411. Google Scholar
Digital Library
- Edya Ladan-Mozes and Nir Shavit. 2008. An optimistic approach to lock-free FIFO queues. Distributed Computing 20, 5 (01 Feb 2008), 323--341.Google Scholar
- Youyou Lu, Jiwu Shu, and Long Sun. 2016. Blurred Persistence: Efficient Transactions in Persistent Memory. Trans. Storage 12, 1, Article 3 (Jan. 2016), 29 pages. Google Scholar
Digital Library
- Managed-Data-Structures 2016. Hewlett Packard Labs: Data structures managed like never before on The Machine. https://www.youtube.com/watch?v=3fN5_Qt9OCs. (2016).Google Scholar
- Maged M. Michael. 2004. Hazard Pointers: Safe Memory Reclamation for Lock-Free Objects. IEEE Trans. Parallel Distrib. Syst. 15, 6 (June 2004), 491--504. Google Scholar
Digital Library
- Maged M. Michael. 2017. Hazard Pointers. https://github.com/facebook/folly/tree/master/folly/experimental/hazptr. (2017).Google Scholar
- Maged M. Michael and Michael L. Scott. 1996. Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms. In Proc. ACM Symposium on Principles of Distributed Computing (PODC). 267--275. Google Scholar
Digital Library
- Iulian Moraru, David G. Andersen, Michael Kaminsky, Nathan Binkert, Niraj Tolia, Reinhard Munz, and Parthasarathy Ranganathan. 2011. Persistent, Protected and Cached: Building Blocks for Main Memory Data Stores. Technical Report CMU-PDL-11-114. Carnegie Mellon University.Google Scholar
- Adam Morrison and Yehuda Afek. 2013. Fast Concurrent Queues for x86 Processors. In Proceedings of the 18th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP '13). ACM, New York, NY, USA, 103--112. Google Scholar
Digital Library
- Steven Pelley, Peter M. Chen, and Thomas F. Wenisch. 2015. Memory Persistency: Semantics for Byte-Addressable Nonvolatile Memory Technologies. IEEE Micro 35, 3 (2015), 125--131.Google Scholar
Digital Library
- RabbitMQ {n. d.}. RabbitMQ - Messaging that works https://www.rabbitmq.com/. ({n. d.}).Google Scholar
- Redis {n. d.}. Redis - in-memory data structure store, http://redis.io/. ({n. d.}).Google Scholar
- Swift {n. d.}. Swift Object Store. https://swift.openstack.org/. ({n. d.}).Google Scholar
- TuxedoMQ {n. d.}. Oracle Tuxedo Message Queue http://www.oracle.com/us/products/middleware/cloud-app-foundation/tuxedo/message-queue/overview/index.html. ({n. d.}).Google Scholar
- Shivaram Venkataraman, Niraj Tolia, Parthasarathy Ranganathan, and Roy H. Campbell. 2011. Consistent and Durable Data Structures for Non-volatile Byte-addressable Memory. In Proceedings of the 9th USENIX Conference on File and Stroage Technologies. 5--5. Google Scholar
Digital Library
- Haris Volos, Andres Jaan Tack, and Michael M. Swift. 2011. Mnemosyne: Lightweight Persistent Memory. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XVI). ACM, New York, NY, USA, 91--104. Google Scholar
Digital Library
- WebSphereMQ {n. d.}. WebSphere MQ - IBM MQ. www.ibm.com/software/products/en/ibm-mq. ({n. d.}).Google Scholar
- Chaoran Yang and John Mellor-Crummey. 2016. A Wait-free Queue As Fast As Fetch-and-add. In Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP '16). ACM, New York, NY, USA, Article 16, 13 pages. Google Scholar
Digital Library
Index Terms
A persistent lock-free queue for non-volatile memory
Recommendations
Mirror: making lock-free data structures persistent
PLDI 2021: Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and ImplementationWith the recent launch of the Intel Optane memory platform, non-volatile main memory in the form of fast, dense, byte-addressable non-volatile memory has now become available. Nevertheless, designing crash-resilient algorithms and data structures is ...
A persistent lock-free queue for non-volatile memory
PPoPP '18: Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel ProgrammingNon-volatile memory is expected to coexist with (or even displace) volatile DRAM for main memory in upcoming architectures. This has led to increasing interest in the problem of designing and specifying durable data structures that can recover from ...
Redesign the Memory Allocator for Non-Volatile Main Memory
Special Issue on Hardware and Algorithms for Learning On-a-chip and Special Issue on Alternative Computing SystemsThe non-volatile memory (NVM) has the merits of byte-addressability, fast speed, persistency and low power consumption, which make it attractive to be used as main memory. Commonly, user process dynamically acquires memory through memory allocators. ...







Comments