Abstract
To ease the development of geo-distributed applications, replicated data types (RDTs) offer a familiar programming interface while ensuring state convergence, low latency, and high availability. However, RDTs are still designed exclusively by experts using ad-hoc solutions that are error-prone and result in brittle systems. Recent works statically detect conflicting operations on existing data types and coordinate those at runtime to guarantee convergence and preserve application invariants. However, these approaches are too conservative, imposing coordination on a large number of operations. In this work, we propose a principled approach to design and implement efficient RDTs taking into account application invariants. Developers extend sequential data types with a distributed specification, which together form an RDT. We statically analyze the specification to detect conflicts and unravel their cause. This information is then used at runtime to serialize concurrent operations safely and efficiently. Our approach derives a correct RDT from any sequential data type without changes to the data type's implementation and with minimal coordination. We implement our approach in Scala and develop an extensive portfolio of RDTs. The evaluation shows that our approach provides performance similar to conflict-free replicated data types for commutative operations, and considerably improves the performance of non-commutative operations, compared to existing solutions.
Supplemental Material
- Paulo Sérgio Almeida, Ali Shoker, and Carlos Baquero. 2015. Efficient State-Based CRDTs by Delta-Mutation. In Networked Systems - Third International Conference, NETYS 2015, Agadir, Morocco, May 13-15, 2015, Revised Selected Papers (Lecture Notes in Computer Science, Vol. 9466). Springer, 62–76. https://doi.org/10.1007/978-3-319-26850-7_5 Google Scholar
Cross Ref
- Peter Bailis, Alan D. Fekete, Michael J. Franklin, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2014. Coordination Avoidance in Database Systems. Proc. VLDB Endow., 8, 3 (2014), 185–196. https://doi.org/10.14778/2735508.2735509 Google Scholar
Digital Library
- Valter Balegas, Sérgio Duarte, Carla Ferreira, Rodrigo Rodrigues, and Nuno M. Preguiça. 2018. IPA: Invariant-preserving Applications for Weakly consistent Replicated Databases. Proc. VLDB Endow., 12, 4 (2018), 404–418. https://doi.org/10.14778/3297753.3297760 Google Scholar
Digital Library
- Valter Balegas, Sérgio Duarte, Carla Ferreira, Rodrigo Rodrigues, Nuno M. Preguiça, Mahsa Najafzadeh, and Marc Shapiro. 2015. Putting consistency back into eventual consistency. In Proceedings of the Tenth European Conference on Computer Systems, EuroSys 2015, Bordeaux, France, April 21-24, 2015. ACM, 6:1–6:16. https://doi.org/10.1145/2741948.2741972 Google Scholar
Digital Library
- Carlos Baquero, Paulo S. Almeida, and Ali Shoker. 2017. Pure Operation-Based Replicated Data Types. CoRR, abs/1710.04469 (2017), arxiv:1710.04469.Google Scholar
- Eric Brewer. 2012. CAP Twelve years later: How the “Rules” have Changed. Computer, 45 (2012), 02, 23–29. https://doi.org/10.1109/MC.2012.37 Google Scholar
Digital Library
- Eric A. Brewer. 2000. Towards robust distributed systems (abstract). In Proceedings of the Nineteenth Annual ACM Symposium on Principles of Distributed Computing, July 16-19, 2000, Portland, Oregon, USA. ACM, 7. https://doi.org/10.1145/343477.343502 Google Scholar
Digital Library
- Sebastian Burckhardt, Manuel Fähndrich, Daan Leijen, and Benjamin P. Wood. 2012. Cloud Types for Eventual Consistency. In ECOOP 2012 - Object-Oriented Programming - 26th European Conference, Beijing, China, June 11-16, 2012. Proceedings (Lecture Notes in Computer Science, Vol. 7313). Springer, 283–307. https://doi.org/10.1007/978-3-642-31057-7_14 Google Scholar
Digital Library
- Emmanuel Cecchet and Julie Marguerite. 2009. RUBiS: Rice University Bidding System. http://rubis.ow2.org/Google Scholar
- Kevin De Porre, Carla Ferreira, Nuno Preguiça, and Elisa Gonzalez Boix. 2021. ECROs: Building Global Scale Systems from Sequential Code (Appendix). http://soft.vub.ac.be/Publications/2021/vub-tr-soft-21-09-appendix.pdfGoogle Scholar
- Kevin De Porre and Elisa Gonzalez Boix. 2019. Squirrel: an extensible distributed key-value store. In Proceedings of the 4th ACM SIGPLAN International Workshop on Meta-Programming Techniques and Reflection, [email protected] 2019, Athens, Greece, October 20, 2019. ACM, 21–30. https://doi.org/10.1145/3358502.3361271 Google Scholar
Digital Library
- Kevin De Porre, Florian Myter, Christophe De Troyer, Christophe Scholliers, Wolfgang De Meuter, and Elisa Gonzalez Boix. 2019. Putting Order in Strong Eventual Consistency. In Distributed Applications and Interoperable Systems - 19th IFIP WG 6.1 International Conference, DAIS 2019, Held as Part of the 14th International Federated Conference on Distributed Computing Techniques, DisCoTec 2019, Kongens Lyngby, Denmark, June 17-21, 2019, Proceedings (Lecture Notes in Computer Science, Vol. 11534). Springer, 36–56. https://doi.org/10.1007/978-3-030-22496-7_3 Google Scholar
Digital Library
- Kevin De Porre, Florian Myter, Christophe Scholliers, and Elisa Gonzalez Boix. 2020. CScript: A distributed programming language for building mixed-consistency applications. J. Parallel Distributed Comput., 144 (2020), 109–123. https://doi.org/10.1016/j.jpdc.2020.05.010 Google Scholar
Cross Ref
- Dimitar I. Dimitrov, Veselin Raychev, Martin T. Vechev, and Eric Koskinen. 2014. Commutativity race detection. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, Edinburgh, United Kingdom - June 09 - 11, 2014. ACM, 305–315. https://doi.org/10.1145/2594291.2594322 Google Scholar
Digital Library
- Alexey Gotsman, Hongseok Yang, Carla Ferreira, Mahsa Najafzadeh, and Marc Shapiro. 2016. ’Cause I’m strong enough: reasoning about consistency choices in distributed systems. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20 - 22, 2016. ACM, 371–384. https://doi.org/10.1145/2837614.2837625 Google Scholar
Digital Library
- Rachid Guerraoui, Matej Pavlovic, and Dragos-Adrian Seredinschi. 2016. Incremental Consistency Guarantees for Replicated Objects. In 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2016, Savannah, GA, USA, November 2-4, 2016. USENIX Association, 169–184. isbn:978-1-931971-33-1 https://www.usenix.org/conference/osdi16/technical-sessions/presentation/guerraouiGoogle Scholar
- Brandon Holt, James Bornholt, Irene Zhang, Dan R. K. Ports, Mark Oskin, and Luis Ceze. 2016. Disciplined Inconsistency with Consistency Types. In Proceedings of the Seventh ACM Symposium on Cloud Computing, Santa Clara, CA, USA, October 5-7, 2016. ACM, 279–293. https://doi.org/10.1145/2987550.2987559 Google Scholar
Digital Library
- Farzin Houshmand and Mohsen Lesani. 2019. Hamsaz: replication coordination analysis and synthesis. Proc. ACM Program. Lang., 3, POPL (2019), 74:1–74:32. https://doi.org/10.1145/3290387 Google Scholar
Digital Library
- Gowtham Kaki, Kapil Earanky, K. C. Sivaramakrishnan, and Suresh Jagannathan. 2018. Safe replication through bounded concurrency verification. Proc. ACM Program. Lang., 2, OOPSLA (2018), 164:1–164:27. https://doi.org/10.1145/3276534 Google Scholar
Digital Library
- Gowtham Kaki, Swarn Priya, K. C. Sivaramakrishnan, and Suresh Jagannathan. 2019. Mergeable replicated data types. Proc. ACM Program. Lang., 3, OOPSLA (2019), 154:1–154:29. https://doi.org/10.1145/3360580 Google Scholar
Digital Library
- Anne-Marie Kermarrec, Antony I. T. Rowstron, Marc Shapiro, and Peter Druschel. 2001. The IceCube approach to the reconciliation of divergent replicas. In Proceedings of the Twentieth Annual ACM Symposium on Principles of Distributed Computing, PODC 2001, Newport, Rhode Island, USA, August 26-29, 2001. ACM, 210–218. https://doi.org/10.1145/383962.384020 Google Scholar
Digital Library
- Martin Kleppmann. 2015. A Critique of the CAP Theorem. ArXiv, abs/1509.05393 (2015).Google Scholar
- Martin Kleppmann and Alastair R. Beresford. 2017. A Conflict-Free Replicated JSON Datatype. IEEE Trans. Parallel Distributed Syst., 28, 10 (2017), 2733–2746. https://doi.org/10.1109/TPDS.2017.2697382 Google Scholar
Digital Library
- Mirko Köhler, Nafise Eskandani, Pascal Weisenburger, Alessandro Margara, and Guido Salvaneschi. 2020. Rethinking safe consistency in distributed object-oriented programming. Proc. ACM Program. Lang., 4, OOPSLA (2020), 188:1–188:30. https://doi.org/10.1145/3428256 Google Scholar
Digital Library
- Milind Kulkarni, Donald Nguyen, Dimitrios Prountzos, Xin Sui, and Keshav Pingali. 2011. Exploiting the commutativity lattice. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2011, San Jose, CA, USA, June 4-8, 2011. ACM, 542–555. https://doi.org/10.1145/1993498.1993562 Google Scholar
Digital Library
- Leslie Lamport. 1978. Time, Clocks, and the Ordering of Events in a Distributed System. Commun. ACM, 21, 7 (1978), 558–565. https://doi.org/10.1145/359545.359563 Google Scholar
Digital Library
- Cheng Li, João Leitão, Allen Clement, Nuno M. Preguiça, Rodrigo Rodrigues, and Viktor Vafeiadis. 2014. Automating the Choice of Consistency Levels in Replicated Systems. In 2014 USENIX Annual Technical Conference, USENIX ATC ’14, Philadelphia, PA, USA, June 19-20, 2014. USENIX Association, 281–292. isbn:978-1-931971-10-2 https://www.usenix.org/conference/atc14/technical-sessions/presentation/li_cheng_2Google Scholar
Digital Library
- Cheng Li, Daniel Porto, Allen Clement, Johannes Gehrke, Nuno M. Preguiça, and Rodrigo Rodrigues. 2012. Making Geo-Replicated Systems Fast as Possible, Consistent when Necessary. In 10th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2012, Hollywood, CA, USA, October 8-10, 2012. USENIX Association, 265–278. isbn:978-1-931971-96-6 https://www.usenix.org/conference/osdi12/technical-sessions/presentation/liGoogle Scholar
- Cheng Li, Nuno M. Preguiça, and Rodrigo Rodrigues. 2018. Fine-grained consistency for geo-replicated systems. In 2018 USENIX Annual Technical Conference, USENIX ATC 2018, Boston, MA, USA, July 11-13, 2018. USENIX Association, 359–372. https://www.usenix.org/conference/atc18/presentation/li-chengGoogle Scholar
- Xiao Li, Farzin Houshmand, and Mohsen Lesani. 2020. Hampa: Solver-Aided Recency-Aware Replication. In Computer Aided Verification - 32nd International Conference, CAV 2020, Los Angeles, CA, USA, July 21-24, 2020, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 12224). Springer, 324–349. https://doi.org/10.1007/978-3-030-53288-8_16 Google Scholar
Cross Ref
- Christopher Meiklejohn and Peter Van Roy. 2015. Lasp: a language for distributed, coordination-free programming. In Proceedings of the 17th International Symposium on Principles and Practice of Declarative Programming, Siena, Italy, July 14-16, 2015. ACM, 184–195. https://doi.org/10.1145/2790449.2790525 Google Scholar
Digital Library
- Matthew Milano and Andrew C. Myers. 2018. MixT: a language for mixing consistency in geodistributed transactions. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2018, Philadelphia, PA, USA, June 18-22, 2018. ACM, 226–241. https://doi.org/10.1145/3192366.3192375 Google Scholar
Digital Library
- Matthew Milano, Rolph Recto, Tom Magrino, and Andrew C. Myers. 2019. A Tour of Gallifrey, a Language for Geodistributed Programming. In 3rd Summit on Advances in Programming Languages, SNAPL 2019, May 16-17, 2019, Providence, RI, USA (LIPIcs, Vol. 136). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 11:1–11:19. https://doi.org/10.4230/LIPIcs.SNAPL.2019.11 Google Scholar
Cross Ref
- Florian Myter, Christophe Scholliers, and Wolfgang De Meuter. 2018. A CAPable distributed programming model. In Proceedings of the 2018 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2018, Boston, MA, USA, November 7-8, 2018. ACM, 88–98. https://doi.org/10.1145/3276954.3276957 Google Scholar
Digital Library
- Sreeja S. Nair, Gustavo Petri, and Marc Shapiro. 2020. Proving the Safety of Highly-Available Distributed Objects. In Programming Languages and Systems - 29th European Symposium on Programming, ESOP 2020, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020, Dublin, Ireland, April 25-30, 2020, Proceedings (Lecture Notes in Computer Science, Vol. 12075). Springer, 544–571. https://doi.org/10.1007/978-3-030-44914-8_20 Google Scholar
Cross Ref
- OpenJDK. [n. d.]. JMH - OpenJDK. https://openjdk.java.net/projects/code-tools/jmh/ Accessed: 13-05-2020.Google Scholar
- David J. Pearce and Paul H. J. Kelly. 2006. A dynamic topological sort algorithm for directed acyclic graphs. ACM J. Exp. Algorithmics, 11 (2006), https://doi.org/10.1145/1187436.1210590 Google Scholar
Digital Library
- Julien Ponge. July 2014. Avoiding Benchmarking Pitfalls on the JVM. https://www.oracle.com/technical-resources/articles/java/architect-benchmarking.html Accessed: 13-05-2020.Google Scholar
- Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. 2011. A comprehensive study of Convergent and Commutative Replicated Data Types. Inria – Centre Paris-Rocquencourt ; INRIA, 50.Google Scholar
- Marc Shapiro, Nuno M. Preguiça, Carlos Baquero, and Marek Zawirski. 2011. Conflict-Free Replicated Data Types. In Stabilization, Safety, and Security of Distributed Systems - 13th International Symposium, SSS 2011, Grenoble, France, October 10-12, 2011. Proceedings (Lecture Notes in Computer Science, Vol. 6976). Springer, 386–400. https://doi.org/10.1007/978-3-642-24550-3_29 Google Scholar
Cross Ref
- K. C. Sivaramakrishnan, Gowtham Kaki, and Suresh Jagannathan. 2015. Declarative programming over eventually consistent data stores. 413–424. https://doi.org/10.1145/2737924.2737981 Google Scholar
Digital Library
- Tim Soethout, Tijs van der Storm, and Jurgen J. Vinju. 2019. Static local coordination avoidance for distributed objects. In Proceedings of the 9th ACM SIGPLAN International Workshop on Programming Based on Actors, Agents, and Decentralized Control, [email protected] 2019, Athens, Greece, October 22, 2019. ACM, 21–30. https://doi.org/10.1145/3358499.3361222 Google Scholar
Digital Library
- Douglas B. Terry, Marvin Theimer, Karin Petersen, Alan J. Demers, Mike Spreitzer, and Carl Hauser. 1995. Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System. In Proceedings of the Fifteenth ACM Symposium on Operating System Principles, SOSP 1995, Copper Mountain Resort, Colorado, USA, December 3-6, 1995. ACM, 172–183. https://doi.org/10.1145/224056.224070 Google Scholar
Digital Library
- Werner Vogels. 2009. Eventually Consistent. Commun. ACM, 52, 1 (2009), 40–44. https://doi.org/10.1145/1435417.1435432 Google Scholar
Digital Library
- Chao Wang, Constantin Enea, Suha Orhun Mutluergil, and Gustavo Petri. 2019. Replication-aware linearizability. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019, Phoenix, AZ, USA, June 22-26, 2019. ACM, 980–993. https://doi.org/10.1145/3314221.3314617 Google Scholar
Digital Library
- Matthew Weidner, Heather Miller, and Christopher Meiklejohn. 2020. Composing and decomposing op-based CRDTs with semidirect products. Proc. ACM Program. Lang., 4, ICFP (2020), 94:1–94:27. https://doi.org/10.1145/3408976 Google Scholar
Digital Library
- Nosheen Zaza and Nathaniel Nystrom. 2016. Data-centric Consistency Policies: A Programming Model for Distributed Applications with Tunable Consistency. In First Workshop on Programming Models and Languages for Distributed Computing, [email protected] 2016, Rome, Italy, July 17, 2016. ACM, 3. https://doi.org/10.1145/2957319.2957377 Google Scholar
Digital Library
- Xin Zhao and Philipp Haller. 2018. Observable atomic consistency for CvRDTs. In Proceedings of the 8th ACM SIGPLAN International Workshop on Programming Based on Actors, Agents, and Decentralized Control, [email protected] 2018, Boston, MA, USA, November 5, 2018. ACM, 23–32. https://doi.org/10.1145/3281366.3281372 Google Scholar
Digital Library
- Xin Zhao and Philipp Haller. 2020. Replicated data types that unify eventual consistency and observable atomic consistency. J. Log. Algebraic Methods Program., 114, 100561. https://doi.org/10.1016/j.jlamp.2020.100561 Google Scholar
Cross Ref
Index Terms
ECROs: building global scale systems from sequential code
Recommendations
Using Paxos to Build a Lightweight, Highly Available Key-Value Data Store
WISA '13: Proceedings of the 2013 10th Web Information System and Application ConferenceKey-value data store has been widely used in e-commerce systems. The availability issue, which means no data loss and continuous service to users, is quite essential in such systems. This paper presents a lightweight, highly available architecture of ...
TARDiS: A Branch-and-Merge Approach To Weak Consistency
SIGMOD '16: Proceedings of the 2016 International Conference on Management of DataThis paper presents the design, implementation, and evaluation of TARDiS (Transactional Asynchronously Replicated Divergent Store), a transactional key-value store explicitly designed for weakly-consistent systems. Reasoning about these systems is hard, ...
Building Replicated Internet Services Using TACT: A Toolkit for Tunable Availability and Consistency Tradeoffs
WECWIS '00: Proceedings of the Second International Workshop on Advance Issues of E-Commerce and Web-Based Information Systems (WECWIS 2000)An ultimate goal for modern Internet services is the development of scalable, high-performance, highly available and fault-tolerant systems. Replication is an important approach to achieve this goal. However, replication introduces the issue of ...






Comments