skip to main content

Modular verification of op-based CRDTs in separation logic

Published:31 October 2022Publication History
Skip Abstract Section

Abstract

Operation-based Conflict-free Replicated Data Types (op-based CRDTs) are a family of distributed data structures where all operations are designed to commute, so that replica states eventually converge. Additionally, op-based CRDTs require that operations be propagated between replicas in causal order. This paper presents a framework for verifying safety properties of CRDT implementations using separation logic. The framework consists of two libraries. One implements a Reliable Causal Broadcast (RCB) protocol so that replicas can exchange messages in causal order. A second “OpLib” library then uses RCB to simplify the creation and correctness proofs of op-based CRDTs. OpLib allows clients to implement new CRDTs as purely-functional data structures, without having to reason about network operations, concurrency control and mutable state, and without having to each time re-implement causal broadcast. Using OpLib, we have implemented 12 example CRDTs from the literature, including multiple versions of replicated registers and sets, two CRDT combinators for products and maps, and two example use cases of the map combinator. Our proofs are conducted in the Aneris distributed separation logic and are formalized in Coq. Our technique is the first work on verification of op-based CRDTs that satisfies both of the following properties: it is modular and targets executable implementations, as opposed to high-level protocols.

References

  1. Mustaque Ahamad, Gil Neiger, James E. Burns, Prince Kohli, and Phillip W. Hutto. 1995. Causal Memory: Definitions, Implementation, and Programming. Distributed Comput., 9, 1 (1995), 37–49. https://doi.org/10.1007/BF01784241 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Peter Bailis, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2013. Bolt-on causal consistency. In Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2013, New York, NY, USA, June 22-27, 2013. 761–772. https://doi.org/10.1145/2463676.2465279 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Carlos Baquero, Paulo Sérgio Almeida, and Ali Shoker. 2014. Making operation-based CRDTs operation-based. In Proceedings of the First Workshop on the Principles and Practice of Eventual Consistency, [email protected] 2014, April 13, 2014, Amsterdam, The Netherlands, Marc Shapiro (Ed.). ACM, 7:1–7:2. https://doi.org/10.1145/2596631.2596632 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Lars Birkedal and Aleš Bizjak. 2017. Lecture Notes on Iris: Higher-Order Concurrent Separation Log. http://iris-project.org/tutorial-pdfs/iris-lecture-notes.pdf Google ScholarGoogle Scholar
  5. Kenneth Birman, Andre Schiper, and Pat Stephenson. 1991. Lightweight Causal and Atomic Group Multicast. ACM Transactions on Computer Systems (TOCS), 9, 3 (1991), 272–314. https://doi.org/10.1145/128738.128742 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Sebastian Burckhardt, Alexey Gotsman, Hongseok Yang, and Marek Zawirski. 2014. Replicated Data Types: Specification, Verification, Optimality. In 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 2014). ACM, 271–284. https://doi.org/10.1145/2535838.2535848 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Christian Cachin, Rachid Guerraoui, and Luís Rodrigues. 2011. Introduction to Reliable and Secure Distributed Programming. Springer Science & Business Media. Google ScholarGoogle Scholar
  8. Tej Chajed, Joseph Tassarotti, M. Frans Kaashoek, and Nickolai Zeldovich. 2019. Verifying concurrent, crash-safe systems with Perennial. In Proceedings of the 27th ACM Symposium on Operating Systems Principles, SOSP 2019, Huntsville, ON, Canada, October 27-30, 2019, Tim Brecht and Carey Williamson (Eds.). ACM, 243–258. https://doi.org/10.1145/3341301.3359632 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Michael Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber. 2008. Bigtable: A Distributed Storage System for Structured Data. ACM Trans. Comput. Syst., 26, 2 (2008), 4:1–4:26. https://doi.org/10.1145/1365815.1365816 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Kristina Chodorow and Michael Dirolf. 2010. MongoDB - The Definitive Guide: Powerful and Scalable Data Storage. O’Reilly. Google ScholarGoogle Scholar
  11. Colin J Fidge. 1987. Timestamps in Message-Passing Systems That Preserve the Partial Ordering. Google ScholarGoogle Scholar
  12. Seth Gilbert and Nancy A. Lynch. 2002. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News, 33, 2 (2002), 51–59. https://doi.org/10.1145/564585.564601 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Victor B. F. Gomes, Martin Kleppmann, Dominic P. Mulligan, and Alastair R. Beresford. 2017. Verifying Strong Eventual Consistency in Distributed Systems. Proc. ACM Program. Lang., 1, OOPSLA (2017), 109:1–109:28. https://doi.org/10.1145/3133933 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Léon Gondelman, Simon Oddershede Gregersen, Abel Nieto, Amin Timany, and Lars Birkedal. 2021. Distributed Causal Memory: Modular Specification and Verification in Higher-Order Distributed Separation Logic. Proc. ACM Program. Lang., 5, POPL (2021), 1–29. https://doi.org/10.1145/3434323 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Maurice Herlihy and Jeannette M. Wing. 1990. Linearizability: A Correctness Condition for Concurrent Objects. ACM Trans. Program. Lang. Syst., 12, 3 (1990), 463–492. https://doi.org/10.1145/78969.78972 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Ralf Jung, Robbert Krebbers, Jacques-Henri Jourdan, Ales Bizjak, Lars Birkedal, and Derek Dreyer. 2018. Iris from the ground up: A modular foundation for higher-order concurrent separation logic. J. Funct. Program., 28 (2018), e20. https://doi.org/10.1017/S0956796818000151 Google ScholarGoogle ScholarCross RefCross Ref
  17. Ralf Jung, David Swasey, Filip Sieczkowski, Kasper Svendsen, Aaron Turon, Lars Birkedal, and Derek Dreyer. 2015. Iris: Monoids and Invariants as an Orthogonal Basis for Concurrent Reasoning. In Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2015, Mumbai, India, January 15-17, 2015. 637–650. https://doi.org/10.1145/2676726.2676980 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Morten Krogh-Jespersen, Amin Timany, Marit Edna Ohlenbusch, Simon Oddershede Gregersen, and Lars Birkedal. 2020. Aneris: A Mechanised Logic for Modular Reasoning about Distributed Systems. In Programming Languages and Systems - 29th European Symposium on Programming, ESOP 2020, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020, Dublin, Ireland, April 25-30, 2020, Proceedings. 336–365. https://doi.org/10.1007/978-3-030-44914-8_13 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Leslie Lamport. 1978. Time, Clocks, and the Ordering of Events in a Distributed System. Commun. ACM, 21, 7 (1978), 558–565. https://doi.org/10.1145/359545.359563 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Adriaan Leijnse, Paulo Sérgio Almeida, and Carlos Baquero. 2019. Higher-Order Patterns in Replicated Data Types. In [email protected]. ACM, 5:1–5:6. https://doi.org/10.1145/3301419.3323971 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Hongjin Liang and Xinyu Feng. 2021. Abstraction for Conflict-Free Replicated Data Types. In PLDI ’21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, Virtual Event, Canada, June 20-25, 20211, Stephen N. Freund and Eran Yahav (Eds.). ACM, 636–650. https://doi.org/10.1145/3453483.3454067 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Yiyun Liu, James Parker, Patrick Redmond, Lindsey Kuper, Michael Hicks, and Niki Vazou. 2020. Verifying Replicated Data Types with Typeclass Refinements in Liquid Haskell. Proc. ACM Program. Lang., 4, OOPSLA (2020), 216:1–216:30. https://doi.org/10.1145/3428284 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Wyatt Lloyd, Michael J. Freedman, Michael Kaminsky, and David G. Andersen. 2011. Don’t settle for eventual: scalable causal consistency for wide-area storage with COPS. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles 2011, SOSP 2011, Cascais, Portugal, October 23-26, 2011. 401–416. https://doi.org/10.1145/2043556.2043593 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Friedemann Mattern. 1988. Virtual Time and Global States of Distributed Systems. Univ., Department of Computer Science. Google ScholarGoogle Scholar
  25. Kartik Nagar and Suresh Jagannathan. 2019. Automated Parameterized Verification of CRDTs. In CAV (2) (Lecture Notes in Computer Science, Vol. 11562). Springer, 459–477. Google ScholarGoogle Scholar
  26. Sreeja S. Nair, Gustavo Petri, and Marc Shapiro. 2020. Proving the Safety of Highly-Available Distributed Objects. In Programming Languages and Systems - 29th European Symposium on Programming, ESOP 2020, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020, Dublin, Ireland, April 25-30, 2020, Proceedings, Peter Müller (Ed.) (Lecture Notes in Computer Science, Vol. 12075). Springer, 544–571. https://doi.org/10.1007/978-3-030-44914-8_20 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Abel Nieto, Léon Gondelman, Alban Reynaud, Amin Timany, and Lars Birkedal. 2022. Modular Verification of Op-Based CRDTs in Separation Logic (Proof Artifact). https://doi.org/10.5281/zenodo.7055010 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Patrick Redmond, Gan Shen, Niki Vazou, and Lindsey Kuper. 2022. Verified Causal Broadcast with Liquid Haskell. arXiv preprint arXiv:2206.14767, https://doi.org/10.48550/arXiv.2206.14767 Google ScholarGoogle Scholar
  29. A. W. Roscoe. 1996. Intensional Specifications of Security Protocols. In CSFW. IEEE Computer Society, 28–38. Google ScholarGoogle Scholar
  30. Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. 2011. A comprehensive study of Convergent and Commutative Replicated Data Types. INRIA. http://hal.inria.fr/inria-00555588/ Google ScholarGoogle Scholar
  31. Marc Shapiro, Nuno M. Preguiça, Carlos Baquero, and Marek Zawirski. 2011. Conflict-Free Replicated Data Types. In Stabilization, Safety, and Security of Distributed Systems - 13th International Symposium, SSS 2011, Grenoble, France, October 10-12, 2011. Proceedings, Xavier Défago, Franck Petit, and Vincent Villain (Eds.) (Lecture Notes in Computer Science, Vol. 6976). Springer, 386–400. https://doi.org/10.1007/978-3-642-24550-3_29 Google ScholarGoogle ScholarCross RefCross Ref
  32. Swaminathan Sivasubramanian. 2012. Amazon dynamoDB: a seamlessly scalable non-relational database service. In Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2012, Scottsdale, AZ, USA, May 20-24, 2012. 729–730. https://doi.org/10.1145/2213836.2213945 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Andrew S. Tanenbaum and Maarten van Steen. 2007. Distributed systems - principles and paradigms, 2nd Edition. Pearson Education. isbn:978-0-13-239227-3 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Amin Timany and Lars Birkedal. 2021. Reasoning about Monotonicity in Separation Logic. In CPP. ACM, 91–104. https://doi.org/10.1145/3437992.3439931 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Amin Timany, Simon Oddershede Gregersen, Léo Stefanesco, Léon Gondelman, Abel Nieto, and Lars Birkedal. 2021. Trillium: Unifying refinement and higher-order distributed separation logic. arXiv preprint arXiv:2109.07863, https://doi.org/10.48550/arXiv.2109.07863 Google ScholarGoogle Scholar
  36. Misha Tyulenev, Andy Schwerin, Asya Kamsky, Randolph Tan, Alyson Cabral, and Jack Mulrow. 2019. Implementation of Cluster-wide Logical Clock and Causal Consistency in MongoDB. In Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, The Netherlands, June 30 - July 5, 2019. 636–650. https://doi.org/10.1145/3299869.3314049 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Niki Vazou, Eric L. Seidel, Ranjit Jhala, Dimitrios Vytiniotis, and Simon L. Peyton Jones. 2014. Refinement Types for Haskell. In ICFP. ACM, 269–282. https://doi.org/10.1145/2628136.2628161 Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Peter Zeller, Annette Bieniusa, and Arnd Poetzsch-Heffter. 2014. Formal Specification and Verification of CRDTs. In FORTE (Lecture Notes in Computer Science, Vol. 8461). Springer, 33–48. https://doi.org/10.1007/978-3-662-43613-4_3 Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Modular verification of op-based CRDTs in separation logic

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!