skip to main content
research-article
Open Access

Sequential programming for replicated data stores

Published:26 July 2019Publication History
Skip Abstract Section

Abstract

We introduce Carol, a refinement-typed programming language for replicated data stores. The salient feature of Carol is that it allows programming and verifying replicated store operations modularly, without consideration of other operations that might interleave, and sequentially, without requiring reference to or knowledge of the concurrent execution model. This is in stark contrast with existing systems, which require understanding the concurrent interactions of all pairs of operations when developing or verifying them.

The key enabling idea is the consistency guard, a two-state predicate relating the locally-viewed store and the hypothetical remote store that an operation’s updates may eventually be applied to, which is used by the Carol programmer to declare their precise consistency requirements. Guards appear to the programmer and refinement typechecker as simple data pre-conditions, enabling sequential reasoning, while appearing to the distributed runtime as consistency control instructions.

We implement and evaluate the Carol system in two parts: (1) the algorithm used to statically translate guards into the runtime coordination actions required to enforce them, and (2) the networked-replica runtime which executes arbitrary operations, written in a Haskell DSL, according to the Carol language semantics.

Skip Supplemental Material Section

Supplemental Material

a106-lewchenko.webm

References

  1. Hagit Attiya, Sebastian Burckhardt, Alexey Gotsman, Adam Morrison, Hongseok Yang, and Marek Zawirski. 2016. Specification and Complexity of Collaborative Text Editing. In Proceedings of the 2016 ACM Symposium on Principles of Distributed Computing (PODC ’16) . ACM, New York, NY, USA, 259–268. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Peter Bailis, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2013. Bolt-on Causal Consistency. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (SIGMOD ’13) . ACM, New York, NY, USA, 761–772. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Valter Balegas, Sérgio Duarte, Carla Ferreira, Rodrigo Rodrigues, and Nuno Preguiça. 2018. IPA: Invariant-preserving Applications for Weakly Consistent Replicated Databases. Proc. VLDB Endow. 12, 4 (Dec. 2018), 404–418. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Valter Balegas, Sérgio Duarte, Carla Ferreira, Rodrigo Rodrigues, Nuno Preguiça, Mahsa Najafzadeh, and Marc Shapiro. 2015. Putting Consistency Back into Eventual Consistency. In Proceedings of the Tenth European Conference on Computer Systems (EuroSys ’15) . ACM, New York, NY, USA, Article 6, 16 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. E. Brewer. 2000. Towards robust distributed systems (abstract). PODC (2000), 7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Sebastian Burckhardt. 2014. Principles of Eventual Consistency. Vol. 1. now publishers. 1–150 pages. https://www.microsoft. com/en-us/research/publication/principles-of-eventual-consistency/ Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Sebastian Burckhardt, Manuel Fahndrich, Daan Leijen, and Mooly Sagiv. 2012. Eventually Consistent Transactions, In Proceedings of the 22n European Symposium on Programming (ESOP). https://www.microsoft.com/en-us/research/ publication/eventually-consistent-transactions/ Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Sebastian Burckhardt, Alexey Gotsman, Hongseok Yang, and Marek Zawirski. 2014. Replicated Data Types: Specification, Verification, Optimality. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’14) . ACM, New York, NY, USA, 271–284. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’08/ETAPS’08) . Springer-Verlag, Berlin, Heidelberg, 337–340. http://dl.acm.org/citation.cfm?id=1792734.1792766 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Seth Gilbert and Nancy Lynch. 2002. Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-tolerant Web Services. SIGACT News 33, 2 (June 2002), 51–59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. Gilbert and N. Lynch. 2012. Perspectives on the CAP Theorem. IEEE Computer 45, 2 (2012), 30–36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Alexey Gotsman, Hongseok Yang, Carla Ferreira, Mahsa Najafzadeh, and Marc Shapiro. 2016. ’Cause I’M Strong Enough: Reasoning About Consistency Choices in Distributed Systems. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’16) . ACM, New York, NY, USA, 371–384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Gowtham Kaki, Kapil Earanky, KC Sivaramakrishnan, and Suresh Jagannathan. 2018. Safe Replication Through Bounded Concurrency Verification. Proc. ACM Program. Lang. 2, OOPSLA, Article 164 (Oct. 2018), 27 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Avinash Lakshman and Prashant Malik. 2010. Cassandra: A Decentralized Structured Storage System. SIGOPS Oper. Syst. Rev. 44, 2 (April 2010), 35–40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Leslie Lamport. 1978. Time, Clocks, and the Ordering of Events in a Distributed System. Commun. ACM 21, 7 (July 1978), 558–565. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Cheng Li, João Leitão, Allen Clement, Nuno Preguiça, Rodrigo Rodrigues, and Viktor Vafeiadis. 2014. Automating the Choice of Consistency Levels in Replicated Systems. In Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference (USENIX ATC’14) . USENIX Association, Berkeley, CA, USA, 281–292. http://dl.acm.org/citation. cfm?id=2643634.2643664 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Cheng Li, Daniel Porto, Allen Clement, Johannes Gehrke, Nuno Preguiça, and Rodrigo Rodrigues. 2012. Making Georeplicated Systems Fast As Possible, Consistent when Necessary. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI’12) . USENIX Association, Berkeley, CA, USA, 265–278. http: //dl.acm.org/citation.cfm?id=2387880.2387906 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Patrick M. Rondon, Ming Kawaguci, and Ranjit Jhala. 2008. Liquid Types. In Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’08) . ACM, New York, NY, USA, 159–169. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Sudip Roy, Lucja Kot, Gabriel Bender, Bailu Ding, Hossein Hojjat, Christoph Koch, Nate Foster, and Johannes Gehrke. 2015. The Homeostasis Protocol: Avoiding Transaction Coordination Through Program Analysis. In SIGMOD. 1311–1326. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. John Rushby, Sam Owre, and Natarajan Shankar. 1998. Subtypes for Specifications: Predicate Subtyping in PVS. IEEE Trans. Softw. Eng. 24, 9 (Sept. 1998), 709–720. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. 2011. Conflict-free Replicated Data Types. In Proceedings of the 13th International Conference on Stabilization, Safety, and Security of Distributed Systems (SSS’11) . Springer-Verlag, Berlin, Heidelberg, 386–400. http://dl.acm.org/citation.cfm?id=2050613.2050642 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. KC Sivaramakrishnan, Gowtham Kaki, and Suresh Jagannathan. 2015. Declarative Programming over Eventually Consistent Data Stores. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15) . ACM, New York, NY, USA, 413–424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Pedro Teixeira. 2017. Decentralized Real-Time Collaborative Documents - Conflict-free editing in the browser using js-ipfs and CRDTs. https://ipfs.io/blog/30-js-ipfs-crdts.md .Google ScholarGoogle Scholar
  24. Douglas B. Terry, Vijayan Prabhakaran, Ramakrishna Kotla, Mahesh Balakrishnan, Marcos K. Aguilera, and Hussam AbuLibdeh. 2013. Consistency-based Service Level Agreements for Cloud Storage. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP ’13) . ACM, New York, NY, USA, 309–324. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Douglas B. Terry, Marvin Theimer, Karin Petersen, Alan J. Demers, Mike Spreitzer, and Carl Hauser. 1995. Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System. In SOSP. 172–183. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Niki Vazou, Eric L. Seidel, Ranjit Jhala, Dimitrios Vytiniotis, and Simon Peyton-Jones. 2014. Refinement Types for Haskell. In Proceedings of the 19th ACM SIGPLAN International Conference on Functional Programming (ICFP ’14). ACM, New York, NY, USA, 269–282. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Hongwei Xi and Frank Pfenning. 1998. Eliminating Array Bound Checking Through Dependent Types. In Proceedings of the ACM SIGPLAN 1998 Conference on Programming Language Design and Implementation (PLDI ’98) . ACM, New York, NY, USA, 249–257. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Sequential programming for replicated data stores

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!