Abstract
Developing and reasoning about systems using eventually consistent data stores is a difficult challenge due to the presence of unexpected behaviors that do not occur under sequential consistency. A fundamental problem in this setting is to identify a correctness criterion that precisely captures intended application behaviors yet is generic enough to be applicable to a wide range of applications.
In this paper, we present such a criterion. More precisely, we generalize conflict serializability to the setting of eventual consistency. Our generalization is based on a novel dependency model that incorporates two powerful algebraic properties: commutativity and absorption. These properties enable precise reasoning about programs that employ high-level replicated data types, common in modern systems. To apply our criterion in practice, we also developed a dynamic analysis algorithm and a tool that checks whether a given program execution is serializable.
We performed a thorough experimental evaluation on two real-world use cases: debugging cloud-backed mobile applications and implementing clients of a popular eventually consistent key-value store. Our experimental results indicate that our criterion reveals harmful synchronization problems in applications, is more effective at finding them than prior approaches, and can be used for the development of practical, eventually consistent applications.
- A. Adya, B. Liskov, and P. E. O’Neil. Generalized isolation level definitions. In Proceedings of the 16th International Conference on Data Engineering, ICDE ’00, pages 67–78, Washington, DC, USA, 2000. IEEE Computer Society. ISBN 0-7695-0506-6. Google Scholar
Digital Library
- P. Bailis, A. Fekete, M. J. Franklin, A. Ghodsi, J. M. Hellerstein, and I. Stoica. Coordination avoidance in database systems. Proc. VLDB Endow., 8(3):185–196, Nov. 2014. ISSN 2150-8097. Google Scholar
Digital Library
- Google Scholar
- G. Bernardi and A. Gotsman. Robustness against consistency models with atomic visibility. In CONCUR’16: International Conference on Concurrency Theory, 2016.Google Scholar
Digital Library
- P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1987. ISBN 0-201-10715-5. Google Scholar
Digital Library
- A. Bouajjani, E. Derevenetc, and R. Meyer. Checking and enforcing robustness against TSO. In Proceedings of the 22nd European Conference on Programming Languages and Systems, ESOP’13, pages 533–553, Berlin, Heidelberg, 2013. Springer-Verlag. ISBN 978- 3-642-37035-9. Google Scholar
Digital Library
- R. Brown, S. Cribbs, C. Meiklejohn, and S. Elliott. Riak DT map: A composable, convergent replicated dictionary. In Proceedings of the First Workshop on Principles and Practice of Eventual Consistency, PaPEC ’14, pages 1:1–1:1, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2716-9. Google Scholar
Digital Library
- S. Burckhardt. Principles of Eventual Consistency, volume 1 of Foundations and Trends in Programming Languages. now publishers, October 2014. Google Scholar
Digital Library
- S. Burckhardt, A. Gotsman, H. Yang, and M. Zawirski. Replicated data types: Specification, verification, optimality. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’14, pages 271–284, New York, NY, USA, 2014. Google Scholar
- ACM. ISBN 978-1-4503-2544-8.Google Scholar
- S. Burckhardt, D. Leijen, J. Protzenko, and M. Fähndrich. Global sequence protocol: A robust abstraction for replicated shared state. In J. T. Boyland, editor, 29th European Conference on Object-Oriented Programming, volume 37 of ECOOP 2015, pages 568–590, Dagstuhl, Germany, 2015. Schloss Dagstuhl– Leibniz-Zentrum fuer Informatik. ISBN 978-3-939897-86-6.Google Scholar
- A. Cerone, G. Bernardi, and A. Gotsman. A Framework for Transactional Consistency Models with Atomic Visibility. In L. Aceto and D. de Frutos Escrig, editors, 26th International Conference on Concurrency Theory, volume 42 of CONCUR 2015, pages 58–71, Dagstuhl, Germany, 2015. Schloss Dagstuhl– Leibniz-Zentrum fuer Informatik. ISBN 978-3-939897-91-0.Google Scholar
Digital Library
- B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni. PNUTS: yahoo!’s hosted data serving platform. Proc. VLDB Endow., 1(2):1277– 1288, Aug. 2008. ISSN 2150-8097. Google Scholar
Digital Library
- G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon’s highly available key-value store. In Proceedings of Twentyfirst ACM SIGOPS Symposium on Operating Systems Principles, SOSP ’07, pages 205–220, New York, NY, USA, 2007. ACM. ISBN 978-1- 59593-591-5. Google Scholar
Digital Library
- D. Dimitrov, V. Raychev, M. Vechev, and E. Koskinen. Commutativity race detection. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, pages 305–315, New York, NY, USA, 2014. ACM. ISBN 978-1-4503- 2784-8. Google Scholar
Digital Library
- A. Fekete, D. Liarokapis, E. O’Neil, P. O’Neil, and D. Shasha. Making snapshot isolation serializable. ACM Trans. Database Syst., 30(2): 492–528, June 2005. ISSN 0362-5915. Google Scholar
- G. Gierz, K. Hofmann, K. Keimel, J. Lawson, M. Mislove, and D. Scott. Continuous Lattices and Domains. Number 93 in Encyclopedia of Mathematics and its Applications. Cambridge University Press, 2003. ISBN 9780521803380.Google Scholar
Digital Library
- S. Gilbert and N. Lynch. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News, 33 (2):51–59, June 2002. ISSN 0163-5700. Google Scholar
Digital Library
- P. Hunt, M. Konar, F. P. Junqueira, and B. Reed. Zookeeper: Waitfree coordination for internet-scale systems. In Proceedings of the 2010 USENIX Conference on USENIX Annual Technical Conference, USENIXATC’10, pages 11–11, Berkeley, CA, USA, 2010. USENIX Association. Google Scholar
Digital Library
- P. R. Johnson and R. H. Thomas. The maintenance of duplicate databases. RFC 677, RFC Editor, january 1975. Google Scholar
Digital Library
- R. Klophaus. Riak core: Building distributed applications without shared state. In ACM SIGPLAN Commercial Users of Functional Programming, CUFP ’10, pages 14:1–14:1, New York, NY, USA, 2010. Google Scholar
- ACM. ISBN 978-1-4503-0516-7.Google Scholar
Digital Library
- A. Lakshman and P. Malik. Cassandra: A decentralized structured storage system. SIGOPS Oper. Syst. Rev., 44(2):35–40, Apr. 2010. ISSN 0163-5980. Google Scholar
Digital Library
- L. Lamport. Time, clocks, and the ordering of events in a distributed system. Commun. ACM, 21(7):558–565, July 1978. ISSN 0001-0782. Google Scholar
Digital Library
- Google Scholar
Digital Library
- L. Lamport. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Trans. Comput., 28(9):690–691, Sept. 1979. ISSN 0018-9340. Google Scholar
Digital Library
- W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Don’t settle for eventual: Scalable causal consistency for wide-area storage with cops. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP ’11, pages 401– 416, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0977-6. Google Scholar
- W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Stronger semantics for low-latency geo-replicated storage. In Proceedings of the 10th USENIX Conference on Networked Systems Design and Implementation, nsdi’13, pages 313–328, Berkeley, CA, USA, 2013. Google Scholar
- USENIX Association.Google Scholar
- F. Mattern. Virtual time and global states of distributed systems. In M. Cosnard, editor, Proc. Workshop on Parallel and Distributed Algorithms, pages 215–226, North-Holland / Elsevier, 1989. (Reprinted in: Z. Yang, T.A. Marsland (Eds.), "Global States and Time in Distributed Systems", IEEE, 1994, pp. 123-133.).Google Scholar
Digital Library
- A. Mazurkiewicz. Trace theory. In W. Brauer, W. Reisig, and G. Rozenberg, editors, Petri Nets: Applications and Relationships to Other Models of Concurrency, volume 255 of Lecture Notes in Computer Science, pages 278–324. Springer Berlin Heidelberg, 1987. ISBN 978-3-540-17906-1.Google Scholar
Digital Library
- S. Owens. Reasoning about the implementation of concurrency abstractions on x86-TSO. In Proceedings of the 24th European Conference on Object-oriented Programming, ECOOP’10, pages 478– 503, Berlin, Heidelberg, 2010. Springer-Verlag. ISBN 3-642-14106-4, 978-3-642-14106-5. Google Scholar
Digital Library
- C. H. Papadimitriou. The serializability of concurrent database updates. J. ACM, 26(4):631–653, Oct. 1979. ISSN 0004-5411. Google Scholar
- Google Scholar
Digital Library
- N. Preguiça, M. Zawirski, A. Bieniusa, S. Duarte, V. Balegas, C. Baquero, and M. Shapiro. SwiftCloud: Fault-tolerant geo-replication integrated all the way to the client machine. In Proceedings of the 2014 IEEE 33rd International Symposium on Reliable Distributed Systems Workshops, SRDSW ’14, pages 30–33, Washington, DC, USA, 2014. IEEE Computer Society. ISBN 978-1-4799-7361-3. Google Scholar
Digital Library
- M. Shapiro, N. Preguiça, C. Baquero, and M. Zawirski. A comprehensive study of Convergent and Commutative Replicated Data Types. Research Report RR-7506, Inria – Centre Paris-Rocquencourt ; INRIA, Jan. 2011.Google Scholar
Digital Library
- M. Shapiro, N. Preguiça, C. Baquero, and M. Zawirski. Conflictfree replicated data types. In Proceedings of the 13th International Conference on Stabilization, Safety, and Security of Distributed Systems, SSS’11, pages 386–400, Berlin, Heidelberg, 2011. Springer-Verlag. ISBN 978-3-642-24549-7. Google Scholar
Digital Library
- D. Shasha and M. Snir. Efficient and correct execution of parallel programs that share memory. ACM Trans. Program. Lang. Syst., 10(2): 282–312, Apr. 1988. ISSN 0164-0925. Google Scholar
Digital Library
- S. Sivasubramanian. Amazon dynamoDB: A seamlessly scalable nonrelational database service. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, SIGMOD ’12, pages 729–730, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1247-9. Google Scholar
- Y. Sovran, R. Power, M. K. Aguilera, and J. Li. Transactional storage for geo-replicated systems. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP ’11, pages 385– 400, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0977-6. Google Scholar
Digital Library
- N. Tillmann, M. Moskal, J. de Halleux, and M. Fahndrich. TouchDevelop: Programming cloud-connected mobile devices via touchscreen. In Proceedings of the 10th SIGPLAN Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2011, pages 49–60, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0941-7. Google Scholar
Digital Library
- Transaction Processing Performance Council. TPC-C benchmark, revision 5.11, Feb. 2010.Google Scholar
Digital Library
- M. Vaziri, F. Tip, and J. Dolby. Associating synchronization constraints with data in an object-oriented language. In Conference Record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’06, pages 334–345, New York, NY, USA, 2006. ACM. ISBN 1-59593-027-2. Google Scholar
- W. E. Weihl. Commutativity-based concurrency control for abstract data types. IEEE Trans. Comput., 37(12):1488–1505, Dec. 1988. ISSN 0018-9340. Google Scholar
- K. Zellag and B. Kemme. How consistent is your cloud application? In Proceedings of the Third ACM Symposium on Cloud Computing, SoCC ’12, pages 6:1–6:14, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1761-0. Google Scholar
Index Terms
Serializability for eventual consistency: criterion, analysis, and applications
Recommendations
Serializability for eventual consistency: criterion, analysis, and applications
POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming LanguagesDeveloping and reasoning about systems using eventually consistent data stores is a difficult challenge due to the presence of unexpected behaviors that do not occur under sequential consistency. A fundamental problem in this setting is to identify a ...
Constraining the eventual in eventual consistency
PaPoC '18: Proceedings of the 5th Workshop on the Principles and Practice of Consistency for Distributed DataCRDTs are highly available replicated data structures which offer strong eventual consistency in the face of concurrent operations [3]. By their definition, CRDTs eventually converge to a consistent state given enough time. However, this is not strict ...
Virtual world consistency: A condition for STM systems (with a versatile protocol with invisible read operations)
The aim of a Software Transactional Memory (STM) is to discharge the programmers from the management of synchronization in multiprocess programs that access concurrent objects. To that end, an STM system provides the programmer with the concept of a ...







Comments