skip to main content
research-article
Open Access

Alone together: compositional reasoning and inference for weak isolation

Published:27 December 2017Publication History
Skip Abstract Section

Abstract

Serializability is a well-understood correctness criterion that simplifies reasoning about the behavior of concurrent transactions by ensuring they are isolated from each other while they execute. However, enforcing serializable isolation comes at a steep cost in performance because it necessarily restricts opportunities to exploit concurrency even when such opportunities would not violate application-specific invariants. As a result, database systems in practice support, and often encourage, developers to implement transactions using weaker alternatives. These alternatives break the strong isolation guarantees offered by serializable transactions to permit greater concurrency. Unfortunately, the semantics of weak isolation is poorly understood, and usually explained only informally in terms of low-level implementation artifacts. Consequently, verifying high-level correctness properties in such environments remains a challenging problem. To address this issue, we present a novel program logic that enables compositional reasoning about the behavior of concurrently executing weakly-isolated transactions. Recognizing that the proof burden necessary to use this logic may dissuade application developers, we also describe an inference procedure based on this foundation that ascertains the weakest isolation level that still guarantees the safety of high-level consistency assertions associated with such transactions. The key to effective inference is the observation that weakly-isolated transactions can be viewed as functional (monadic) computations over an abstract database state, allowing us to treat their operations as state transformers over the database. This interpretation enables automated verification using off-the-shelf SMT solvers. Our development is parametric over a transaction’s specific isolation semantics, allowing it to be applicable over a range of concurrency control mechanisms. Case studies and experiments on real-world applications (written in an embedded DSL in OCaml) demonstrate the utility of our approach, and provide strong evidence that automated verification of weakly-isolated transactions can be placed on the same formal footing as their strongly-isolated serializable counterparts.

Skip Supplemental Material Section

Supplemental Material

alonetogether.webm

References

  1. Atul Adya. 1999. Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions. Ph.D. Dissertation. Cambridge, MA, USA. AAI0800775.Google ScholarGoogle Scholar
  2. Peter Alvaro, Neil Conway, Joe Hellerstein, and William R. Marczak. 2011. Consistency Analysis in Bloom: a CALM and Collected Approach. In CIDR 2011, Fifth Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 9-12, 2011, Online Proceedings. 249–260.Google ScholarGoogle Scholar
  3. Timothy G. Armstrong, Vamsi Ponnekanti, Dhruba Borthakur, and Mark Callaghan. 2013. LinkBench: A Database Benchmark Based on the Facebook Social Graph. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (SIGMOD ’13). ACM, New York, NY, USA, 1185–1196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Peter Bailis, Aaron Davidson, Alan Fekete, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2013a. Highly Available Transactions: Virtues and Limitations. PVLDB 7, 3 (2013), 181–192.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Peter Bailis, Alan Fekete, Michael J. Franklin, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2014. Coordination Avoidance in Database Systems. Proc. VLDB Endow. 8, 3 (Nov. 2014), 185–196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Peter Bailis, Alan Fekete, Michael J. Franklin, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2015. Feral Concurrency Control: An Empirical Investigation of Modern Application Integrity. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data (SIGMOD ’15). ACM, New York, NY, USA, 1327–1342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Peter Bailis, Alan Fekete, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2013b. HAT, Not CAP: Towards Highly Available Transactions. In Proceedings of the 14th USENIX Conference on Hot Topics in Operating Systems (HotOS’13). USENIX Association, Berkeley, CA, USA, 24–24. http://dl.acm.org/citation.cfm?id=2490483.2490507Google ScholarGoogle Scholar
  8. Valter Balegas, Nuno Preguiça, Rodrigo Rodrigues, Sérgio Duarte, Carla Ferreira, Mahsa Najafzadeh, and Marc Shapiro. 2015. Putting the Consistency back into Eventual Consistency. In Proceedings of the Tenth European Conference on Computer System (EuroSys ’15). Bordeaux, France. http://lip6.fr/Marc.Shapiro/papers/putting- consistency- back- EuroSys- 2015.pdfGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  9. Hal Berenson, Phil Bernstein, Jim Gray, Jim Melton, Elizabeth O’Neil, and Patrick O’Neil. 1995. A Critique of ANSI SQL Isolation Levels. In Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data (SIGMOD ’95). ACM, New York, NY, USA, 1–10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Arthur J. Bernstein, Philip M. Lewis, and Shiyong Lu. 2000. Semantic Conditions for Correctness at Different Isolation Levels. In Proceedings of the 16th International Conference on Data Engineering (ICDE ’00). IEEE Computer Society, Washington, DC, USA. http://dl.acm.org/citation.cfm?id=846219.847381 Google ScholarGoogle ScholarCross RefCross Ref
  11. Philip A. Bernstein and Sudipto Das. 2013. Rethinking Eventual Consistency. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (SIGMOD ’13). ACM, New York, NY, USA, 923–928. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Philip A. Bernstein and Nathan Goodman. 1983. Multiversion Concurrency Control - Theory and Algorithms. ACM Trans. Database Syst. 8, 4 (Dec. 1983), 465–483. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Bitcoin Bug 2016. How I Stole Roughly 100 BTC From an Exchange and How I Could Have Stolen More! https://goo.gl/4SqaP2Google ScholarGoogle Scholar
  14. Ergon Börger, Erich Grädel, and Yuri Gurevich. 1996. The Classical Decision Problem. Springer-Verlag Telos.Google ScholarGoogle Scholar
  15. Sebastian Burckhardt, Alexey Gotsman, Hongseok Yang, and Marek Zawirski. 2014. Replicated Data Types: Specification, Verification, Optimality. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’14). ACM, New York, NY, USA, 271–284. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Michael J. Cahill, Uwe Röhm, and Alan D. Fekete. 2008. Serializable Isolation for Snapshot Databases. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data (SIGMOD ’08). ACM, New York, NY, USA, 729–738. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Andrea Cerone, Giovanni Bernardi, and Alexey Gotsman. 2015. A Framework for Transactional Consistency Models with Atomic Visibility. In 26th International Conference on Concurrency Theory (CONCUR 2015) (Leibniz International Proceedings in Informatics (LIPIcs)), Luca Aceto and David de Frutos Escrig (Eds.), Vol. 42. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 58–71. Google ScholarGoogle ScholarCross RefCross Ref
  18. Andrea Cerone and Alexey Gotsman. 2016. Analysing Snapshot Isolation. In Proceedings of the 2016 ACM Symposium on Principles of Distributed Computing (PODC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Natacha Crooks, Youer Pu, Lorenzo Alvisi, and Allen Clement. 2017. Seeing is Believing: A Client-Centric Specification of Database Isolation. In Proceedings of the ACM Conference on Principles of Distributed Computing (PODC). 73–82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Susan B. Davidson, Hector Garcia-Molina, and Dale Skeen. 1985. Consistency in a Partitioned Network: A Survey. ACM Comput. Surv. 17, 3 (Sept. 1985), 341–370. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. K. P. Eswaran, J. N. Gray, R. A. Lorie, and I. L. Traiger. 1976. The Notions of Consistency and Predicate Locks in a Database System. Commun. ACM 19, 11 (Nov. 1976), 624–633. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Alan Fekete, Shirley N. Goldrei, and Jorge Pérez Asenjo. 2009. Quantifying Isolation Anomalies. Proc. VLDB Endow. 2, 1 (Aug. 2009), 467–478. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Alan Fekete, Dimitrios Liarokapis, Elizabeth O’Neil, Patrick O’Neil, and Dennis Shasha. 2005. Making Snapshot Isolation Serializable. ACM Trans. Database Syst. 30, 2 (June 2005), 492–528. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Peter Gammie, Antony L. Hosking, and Kai Engelhardt. 2015. Relaxing Safely: Verified On-the-fly Garbage Collection for x86-TSO. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15). ACM, New York, NY, USA, 99–109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Hector Garcia-Molina, Jeffrey D. Ullman, and Jennifer Widom. 2008. Database Systems: The Complete Book (2 ed.). Prentice Hall Press, Upper Saddle River, NJ, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Seth Gilbert and Nancy Lynch. 2002. Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-tolerant Web Services. SIGACT News 33, 2 (June 2002), 51–59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Alexey Gotsman, Hongseok Yang, Carla Ferreira, Mahsa Najafzadeh, and Marc Shapiro. 2016. ’Cause I’m Strong Enough: Reasoning About Consistency Choices in Distributed Systems. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 2016). ACM, New York, NY, USA, 371–384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. J. N. Gray, R. A. Lorie, G. R. Putzolu, and I. L. Traiger. 1976. Granularity of Locks and Degrees of Consistency in a Shared Data Base. 365–394.Google ScholarGoogle Scholar
  29. Chris Hawblitzel, Erez Petrank, Shaz Qadeer, and Serdar Tasiran. 2015. Automated and Modular Refinement Reasoning for Concurrent Programs. In Computer Aided Verification: 27th International Conference. Springer International Publishing, 449–465. Google ScholarGoogle ScholarCross RefCross Ref
  30. C. B. Jones. 1983. Tentative Steps Toward a Development Method for Interfering Programs. ACM Trans. Program. Lang. Syst. 5, 4 (Oct. 1983), 596–619. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Gowtham Kaki, Kartik Nagar, Mahsa Najafzadeh, and Suresh Jagannathan. 2018. Alone Together: Compositional Reasoning and Inference for Weak Isolation. https://arxiv.org/abs/1710.09844.Google ScholarGoogle Scholar
  32. Mohsen Lesani, Christian J. Bell, and Adam Chlipala. 2016. Chapar: Certified Causally Consistent Distributed Key-value Stores. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’16). ACM, New York, NY, USA, 357–370. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Cheng Li, João Leitão, Allen Clement, Nuno Preguiça, Rodrigo Rodrigues, and Viktor Vafeiadis. 2014. Automating the Choice of Consistency Levels in Replicated Systems. In Proceedings of USENIX Annual Technical Conference (USENIX ATC’14). USENIX Association, Berkeley, CA, USA, 281–292. http://dl.acm.org/citation.cfm?id=2643634.2643664Google ScholarGoogle Scholar
  34. Cheng Li, Daniel Porto, Allen Clement, Johannes Gehrke, Nuno Preguiça, and Rodrigo Rodrigues. 2012. Making Georeplicated Systems Fast As Possible, Consistent when Necessary. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI’12). USENIX Association, Berkeley, CA, USA, 265–278. http: //dl.acm.org/citation.cfm?id=2387880.2387906Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. MySQL 2016. Transaction Isolation Levels. https://dev.mysql.com/doc/refman/5.6/en/innodb- transaction- isolation- levels. html Accessed: 2016-07-1 10:00:00.Google ScholarGoogle Scholar
  36. Oracle 2016. Data Concurrency and Consistency. https://docs.oracle.com/cd/B28359_01/server.111/b28318/consist.htm Accessed: 2016-07-1 10:00:00.Google ScholarGoogle Scholar
  37. Poloniex Bug 2016. BTC Stolen from Poloniex. https://bitcointalk.org/index.php?topic=499580Google ScholarGoogle Scholar
  38. PostgreSQL 2016. Transaction Isolation. https://www.postgresql.org/docs/9.1/static/transaction- iso.html Accessed: 2016-07-1 10:00:00.Google ScholarGoogle Scholar
  39. Stephen Revilak, Patrick O’Neil, and Elizabeth O’Neil. 2011. Precisely Serializable Snapshot Isolation (PSSI). In Proceedings of the 2011 IEEE 27th International Conference on Data Engineering (ICDE ’11). IEEE Computer Society, Washington, DC, USA, 482–493. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. J C Reynolds. 2002. Separation Logic: A Logic for Shared Mutable Data Structures. In 17 t h Annual IEEE Symposium on Logic in Computer Science. IEEE Comput. Soc, 55–74. Google ScholarGoogle ScholarCross RefCross Ref
  41. SciMed Bug 2016. Avoid Race Conditions that Violate Uniqueness Validation - Rails. http://goo.gl/0QhMQjGoogle ScholarGoogle Scholar
  42. Dennis Shasha and Philippe Bonnet. 2003. Database Tuning: Principles, Experiments, and Troubleshooting Techniques. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.Google ScholarGoogle Scholar
  43. KC Sivaramakrishnan, Gowtham Kaki, and Suresh Jagannathan. 2015. Declarative Programming over Eventually Consistent Data Stores. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2015). ACM, New York, NY, USA, 413–424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Yair Sovran, Russell Power, Marcos K. Aguilera, and Jinyang Li. 2011. Transactional Storage for Geo-replicated Systems. In Proceedings of the 23 r d ACM Symposium on Operating Systems Principles (SOSP ’11). ACM, New York, NY, USA, 385–400. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Starbucks Bug 2016. Hacking Starbucks for unlimited coffee. http://sakurity.com/blog/2015/05/21/starbucks.htmlGoogle ScholarGoogle Scholar
  46. Viktor Vafeiadis. 2010. Automatically Proving Linearizability. In Proceedings of the 22 nd International Conference on Computer Aided Verification (CAV’10). Springer-Verlag, Berlin, Heidelberg, 450–464. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Vafeiadis, Viktor. 2010. RGSep Action Inference. In Proceedings of the International Conference on Verification, Model Checking, and Abstract Interpretation. 345–361. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Vafeiadis, Viktor and Parkinson, Matthew. 2007. A Marriage of Rely/Guarantee and Separation Logic. In CONCUR 2007 – Concurrency Theory. Springer Berlin Heidelberg, Berlin, Heidelberg, 256–271.Google ScholarGoogle Scholar
  49. Todd Warszawski and Peter Bailis. 2017. ACIDRain: Concurrency-Related Attacks on Database-Backed Web Applications. In Proceedings of the 2017 ACM International Conference on Management of Data (SIGMOD ’17). ACM, New York, NY, USA, 5–20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Kamal Zellag and Bettina Kemme. 2014. Consistency Anomalies in Multi-tier Architectures: Automatic Detection and Prevention. The VLDB Journal 23, 1 (Feb. 2014), 147–172. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Alone together: compositional reasoning and inference for weak isolation

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image Proceedings of the ACM on Programming Languages
          Proceedings of the ACM on Programming Languages  Volume 2, Issue POPL
          January 2018
          1961 pages
          EISSN:2475-1421
          DOI:10.1145/3177123
          Issue’s Table of Contents

          Copyright © 2017 Owner/Author

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 27 December 2017
          Published in pacmpl Volume 2, Issue POPL

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!