skip to main content
research-article

Serializability for eventual consistency: criterion, analysis, and applications

Published:01 January 2017Publication History
Skip Abstract Section

Abstract

Developing and reasoning about systems using eventually consistent data stores is a difficult challenge due to the presence of unexpected behaviors that do not occur under sequential consistency. A fundamental problem in this setting is to identify a correctness criterion that precisely captures intended application behaviors yet is generic enough to be applicable to a wide range of applications.

In this paper, we present such a criterion. More precisely, we generalize conflict serializability to the setting of eventual consistency. Our generalization is based on a novel dependency model that incorporates two powerful algebraic properties: commutativity and absorption. These properties enable precise reasoning about programs that employ high-level replicated data types, common in modern systems. To apply our criterion in practice, we also developed a dynamic analysis algorithm and a tool that checks whether a given program execution is serializable.

We performed a thorough experimental evaluation on two real-world use cases: debugging cloud-backed mobile applications and implementing clients of a popular eventually consistent key-value store. Our experimental results indicate that our criterion reveals harmful synchronization problems in applications, is more effective at finding them than prior approaches, and can be used for the development of practical, eventually consistent applications.

References

  1. A. Adya, B. Liskov, and P. E. O’Neil. Generalized isolation level definitions. In Proceedings of the 16th International Conference on Data Engineering, ICDE ’00, pages 67–78, Washington, DC, USA, 2000. IEEE Computer Society. ISBN 0-7695-0506-6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. P. Bailis, A. Fekete, M. J. Franklin, A. Ghodsi, J. M. Hellerstein, and I. Stoica. Coordination avoidance in database systems. Proc. VLDB Endow., 8(3):185–196, Nov. 2014. ISSN 2150-8097. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Google ScholarGoogle Scholar
  4. G. Bernardi and A. Gotsman. Robustness against consistency models with atomic visibility. In CONCUR’16: International Conference on Concurrency Theory, 2016.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1987. ISBN 0-201-10715-5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. A. Bouajjani, E. Derevenetc, and R. Meyer. Checking and enforcing robustness against TSO. In Proceedings of the 22nd European Conference on Programming Languages and Systems, ESOP’13, pages 533–553, Berlin, Heidelberg, 2013. Springer-Verlag. ISBN 978- 3-642-37035-9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. R. Brown, S. Cribbs, C. Meiklejohn, and S. Elliott. Riak DT map: A composable, convergent replicated dictionary. In Proceedings of the First Workshop on Principles and Practice of Eventual Consistency, PaPEC ’14, pages 1:1–1:1, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2716-9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. S. Burckhardt. Principles of Eventual Consistency, volume 1 of Foundations and Trends in Programming Languages. now publishers, October 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. S. Burckhardt, A. Gotsman, H. Yang, and M. Zawirski. Replicated data types: Specification, verification, optimality. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’14, pages 271–284, New York, NY, USA, 2014. Google ScholarGoogle Scholar
  10. ACM. ISBN 978-1-4503-2544-8.Google ScholarGoogle Scholar
  11. S. Burckhardt, D. Leijen, J. Protzenko, and M. Fähndrich. Global sequence protocol: A robust abstraction for replicated shared state. In J. T. Boyland, editor, 29th European Conference on Object-Oriented Programming, volume 37 of ECOOP 2015, pages 568–590, Dagstuhl, Germany, 2015. Schloss Dagstuhl– Leibniz-Zentrum fuer Informatik. ISBN 978-3-939897-86-6.Google ScholarGoogle Scholar
  12. A. Cerone, G. Bernardi, and A. Gotsman. A Framework for Transactional Consistency Models with Atomic Visibility. In L. Aceto and D. de Frutos Escrig, editors, 26th International Conference on Concurrency Theory, volume 42 of CONCUR 2015, pages 58–71, Dagstuhl, Germany, 2015. Schloss Dagstuhl– Leibniz-Zentrum fuer Informatik. ISBN 978-3-939897-91-0.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni. PNUTS: yahoo!’s hosted data serving platform. Proc. VLDB Endow., 1(2):1277– 1288, Aug. 2008. ISSN 2150-8097. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon’s highly available key-value store. In Proceedings of Twentyfirst ACM SIGOPS Symposium on Operating Systems Principles, SOSP ’07, pages 205–220, New York, NY, USA, 2007. ACM. ISBN 978-1- 59593-591-5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. D. Dimitrov, V. Raychev, M. Vechev, and E. Koskinen. Commutativity race detection. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, pages 305–315, New York, NY, USA, 2014. ACM. ISBN 978-1-4503- 2784-8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Fekete, D. Liarokapis, E. O’Neil, P. O’Neil, and D. Shasha. Making snapshot isolation serializable. ACM Trans. Database Syst., 30(2): 492–528, June 2005. ISSN 0362-5915. Google ScholarGoogle Scholar
  17. G. Gierz, K. Hofmann, K. Keimel, J. Lawson, M. Mislove, and D. Scott. Continuous Lattices and Domains. Number 93 in Encyclopedia of Mathematics and its Applications. Cambridge University Press, 2003. ISBN 9780521803380.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. S. Gilbert and N. Lynch. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News, 33 (2):51–59, June 2002. ISSN 0163-5700. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. P. Hunt, M. Konar, F. P. Junqueira, and B. Reed. Zookeeper: Waitfree coordination for internet-scale systems. In Proceedings of the 2010 USENIX Conference on USENIX Annual Technical Conference, USENIXATC’10, pages 11–11, Berkeley, CA, USA, 2010. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. P. R. Johnson and R. H. Thomas. The maintenance of duplicate databases. RFC 677, RFC Editor, january 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. R. Klophaus. Riak core: Building distributed applications without shared state. In ACM SIGPLAN Commercial Users of Functional Programming, CUFP ’10, pages 14:1–14:1, New York, NY, USA, 2010. Google ScholarGoogle Scholar
  22. ACM. ISBN 978-1-4503-0516-7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. A. Lakshman and P. Malik. Cassandra: A decentralized structured storage system. SIGOPS Oper. Syst. Rev., 44(2):35–40, Apr. 2010. ISSN 0163-5980. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. L. Lamport. Time, clocks, and the ordering of events in a distributed system. Commun. ACM, 21(7):558–565, July 1978. ISSN 0001-0782. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. L. Lamport. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Trans. Comput., 28(9):690–691, Sept. 1979. ISSN 0018-9340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Don’t settle for eventual: Scalable causal consistency for wide-area storage with cops. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP ’11, pages 401– 416, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0977-6. Google ScholarGoogle Scholar
  28. W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Stronger semantics for low-latency geo-replicated storage. In Proceedings of the 10th USENIX Conference on Networked Systems Design and Implementation, nsdi’13, pages 313–328, Berkeley, CA, USA, 2013. Google ScholarGoogle Scholar
  29. USENIX Association.Google ScholarGoogle Scholar
  30. F. Mattern. Virtual time and global states of distributed systems. In M. Cosnard, editor, Proc. Workshop on Parallel and Distributed Algorithms, pages 215–226, North-Holland / Elsevier, 1989. (Reprinted in: Z. Yang, T.A. Marsland (Eds.), "Global States and Time in Distributed Systems", IEEE, 1994, pp. 123-133.).Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. A. Mazurkiewicz. Trace theory. In W. Brauer, W. Reisig, and G. Rozenberg, editors, Petri Nets: Applications and Relationships to Other Models of Concurrency, volume 255 of Lecture Notes in Computer Science, pages 278–324. Springer Berlin Heidelberg, 1987. ISBN 978-3-540-17906-1.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. S. Owens. Reasoning about the implementation of concurrency abstractions on x86-TSO. In Proceedings of the 24th European Conference on Object-oriented Programming, ECOOP’10, pages 478– 503, Berlin, Heidelberg, 2010. Springer-Verlag. ISBN 3-642-14106-4, 978-3-642-14106-5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. C. H. Papadimitriou. The serializability of concurrent database updates. J. ACM, 26(4):631–653, Oct. 1979. ISSN 0004-5411. Google ScholarGoogle Scholar
  34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. N. Preguiça, M. Zawirski, A. Bieniusa, S. Duarte, V. Balegas, C. Baquero, and M. Shapiro. SwiftCloud: Fault-tolerant geo-replication integrated all the way to the client machine. In Proceedings of the 2014 IEEE 33rd International Symposium on Reliable Distributed Systems Workshops, SRDSW ’14, pages 30–33, Washington, DC, USA, 2014. IEEE Computer Society. ISBN 978-1-4799-7361-3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. M. Shapiro, N. Preguiça, C. Baquero, and M. Zawirski. A comprehensive study of Convergent and Commutative Replicated Data Types. Research Report RR-7506, Inria – Centre Paris-Rocquencourt ; INRIA, Jan. 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. M. Shapiro, N. Preguiça, C. Baquero, and M. Zawirski. Conflictfree replicated data types. In Proceedings of the 13th International Conference on Stabilization, Safety, and Security of Distributed Systems, SSS’11, pages 386–400, Berlin, Heidelberg, 2011. Springer-Verlag. ISBN 978-3-642-24549-7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. D. Shasha and M. Snir. Efficient and correct execution of parallel programs that share memory. ACM Trans. Program. Lang. Syst., 10(2): 282–312, Apr. 1988. ISSN 0164-0925. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. S. Sivasubramanian. Amazon dynamoDB: A seamlessly scalable nonrelational database service. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, SIGMOD ’12, pages 729–730, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1247-9. Google ScholarGoogle Scholar
  40. Y. Sovran, R. Power, M. K. Aguilera, and J. Li. Transactional storage for geo-replicated systems. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP ’11, pages 385– 400, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0977-6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. N. Tillmann, M. Moskal, J. de Halleux, and M. Fahndrich. TouchDevelop: Programming cloud-connected mobile devices via touchscreen. In Proceedings of the 10th SIGPLAN Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2011, pages 49–60, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0941-7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Transaction Processing Performance Council. TPC-C benchmark, revision 5.11, Feb. 2010.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. M. Vaziri, F. Tip, and J. Dolby. Associating synchronization constraints with data in an object-oriented language. In Conference Record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’06, pages 334–345, New York, NY, USA, 2006. ACM. ISBN 1-59593-027-2. Google ScholarGoogle Scholar
  44. W. E. Weihl. Commutativity-based concurrency control for abstract data types. IEEE Trans. Comput., 37(12):1488–1505, Dec. 1988. ISSN 0018-9340. Google ScholarGoogle Scholar
  45. K. Zellag and B. Kemme. How consistent is your cloud application? In Proceedings of the Third ACM Symposium on Cloud Computing, SoCC ’12, pages 6:1–6:14, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1761-0. Google ScholarGoogle Scholar

Index Terms

  1. Serializability for eventual consistency: criterion, analysis, and applications

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 52, Issue 1
          POPL '17
          January 2017
          901 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/3093333
          Issue’s Table of Contents
          • cover image ACM Conferences
            POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages
            January 2017
            901 pages
            ISBN:9781450346603
            DOI:10.1145/3009837

          Copyright © 2017 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 1 January 2017

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!