skip to main content
research-article
Open Access

CLOTHO: directed test generation for weakly consistent database systems

Published:10 October 2019Publication History
Skip Abstract Section

Abstract

Relational database applications are notoriously difficult to test and debug. Concurrent execution of database transactions may violate complex structural invariants that constraint how changes to the contents of one (shared) table affect the contents of another. Simplifying the underlying concurrency model is one way to ameliorate the difficulty of understanding how concurrent accesses and updates can affect database state with respect to these sophisticated properties. Enforcing serializable execution of all transactions achieves this simplification, but it comes at a significant price in performance, especially at scale, where database state is often replicated to improve latency and availability.

To address these challenges, this paper presents a novel testing framework for detecting serializability violations in (SQL) database-backed Java applications executing on weakly-consistent storage systems. We manifest our approach in a tool, CLOTHO, that combines a static analyzer and model checker to generate abstract executions, discover serializability violations in these executions, and translate them back into concrete test inputs suitable for deployment in a test environment. To the best of our knowledge, CLOTHO, is the first automated test generation facility for identifying serializability anomalies of Java applications intended to operate in geo-replicated distributed environments. An experimental evaluation on a set of industry-standard benchmarks demonstrates the utility of our approach.

Skip Supplemental Material Section

Supplemental Material

a117-rahmani

Presentation at OOPSLA '19

References

  1. 2010. TPC-C Benchmark. http://www.tpc.org/tpc_documents_current_versions/pdf/tpc-c_v5.11.0.pdf . Online; Accessed April 2019.Google ScholarGoogle Scholar
  2. Atul Adya. 1999. Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions. Ph.D. Dissertation. Cambridge, MA, USA. AAI0800775.Google ScholarGoogle Scholar
  3. Atul Adya, Barbara Liskov, and Patrick E. O’Neil. 2000. Generalized Isolation Level Definitions. In Proceedings of the 16th International Conference on Data Engineering, San Diego, California, USA, February 28 - March 3, 2000 . 67–78. Google ScholarGoogle Scholar
  4. Peter Alvaro, Neil Conway, Joe Hellerstein, and William R. Marczak. 2011. Consistency Analysis in Bloom: a CALM and Collected Approach. In CIDR 2011, Fifth Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 9-12, 2011, Online Proceedings . 249–260.Google ScholarGoogle Scholar
  5. Peter Bailis, Alan Fekete, Michael J. Franklin, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2014a. Coordination Avoidance in Database Systems. Proc. VLDB Endow. 8, 3 (Nov. 2014), 185–196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Peter Bailis, Alan Fekete, Joseph M. Hellerstein, Ali Ghodsi, and Ion Stoica. 2014b. Scalable Atomic Visibility with RAMP Transactions. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data (SIGMOD ’14). ACM, New York, NY, USA, 27–38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Valter Balegas, Nuno Preguiça, Rodrigo Rodrigues, Sérgio Duarte, Carla Ferreira, Mahsa Najafzadeh, and Marc Shapiro. 2015. Putting the Consistency back into Eventual Consistency. In Proceedings of the Tenth European Conference on Computer System (EuroSys ’15) . Bordeaux, France. http://lip6.fr/Marc.Shapiro/papers/putting-consistency-back-EuroSys-2015.pdfGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  8. Hal Berenson, Phil Bernstein, Jim Gray, Jim Melton, Elizabeth O’Neil, and Patrick O’Neil. 1995. A Critique of ANSI SQL Isolation Levels. In Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data (SIGMOD ’95). ACM, New York, NY, USA, 1–10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Giovanni Bernardi and Alexey Gotsman. 2016. Robustness against Consistency Models with Atomic Visibility. In 27th International Conference on Concurrency Theory, CONCUR 2016, August 23-26, 2016, Québec City, Canada . 7:1–7:15. Google ScholarGoogle ScholarCross RefCross Ref
  10. Blockade. 2019. Blockade. https://github.com/worstcase/blockade accessed on 2019-03-29.Google ScholarGoogle Scholar
  11. Lucas Brutschy, Dimitar Dimitrov, Peter Müller, and Martin T. Vechev. 2017. Serializability for eventual consistency: criterion, analysis, and applications. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL 2017, Paris, France, January 18-20, 2017 . 458–472. http://dl.acm.org/citation.cfm?id=3009895Google ScholarGoogle Scholar
  12. Lucas Brutschy, Dimitar Dimitrov, Peter Müller, and Martin T. Vechev. 2018. Static serializability analysis for causal consistency. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2018, Philadelphia, PA, USA, June 18-22, 2018 . 90–104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Sebastian Burckhardt, Alexey Gotsman, Hongseok Yang, and Marek Zawirski. 2014. Replicated Data Types: Specification, Verification, Optimality. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’14) . ACM, New York, NY, USA, 271–284. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Andrea Cerone, Giovanni Bernardi, and Alexey Gotsman. 2015. A Framework for Transactional Consistency Models with Atomic Visibility. In 26th International Conference on Concurrency Theory, CONCUR 2015, Madrid, Spain, September 1.4, 2015 . 58–71. Google ScholarGoogle Scholar
  15. Andrea Cerone and Alexey Gotsman. 2016. Analysing Snapshot Isolation. In Proceedings of the 2016 ACM Symposium on Principles of Distributed Computing, PODC 2016, Chicago, IL, USA, July 25-28, 2016 . 55–64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Andrea Cerone, Alexey Gotsman, and Hongseok Yang. 2017. Algebraic Laws for Weak Consistency. In 28th International Conference on Concurrency Theory, CONCUR 2017, September 5-8, 2017, Berlin, Germany . 26:1–26:18. Google ScholarGoogle ScholarCross RefCross Ref
  17. Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Tools and Algorithms for the Construction and Analysis of Systems , C. R. Ramakrishnan and Jakob Rehof (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 337–340.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Djellel Eddine Difallah, Andrew Pavlo, Carlo Curino, and Philippe Cudre-Mauroux. 2013. OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases. Proc. VLDB Endow. 7, 4 (Dec. 2013), 277–288. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Alan Fekete, Dimitrios Liarokapis, Elizabeth J. O’Neil, and Patrick E. O’Neil a fnd Dennis E. Shasha. 2005. Making snapshot isolation serializable. ACM Trans. Database Syst. 30, 2 (2005), 492–528. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Cormac Flanagan, Cormac Flanagan, and Stephen N Freund. 2004. Atomizer: A Dynamic Atomicity Checker for Multithreaded Programs. In Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’04) . ACM, New York, NY, USA, 256–267. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Cormac Flanagan and Shaz Qadeer. 2003. A Type and Effect System for Atomicity. In Proceedings of the ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation (PLDI ’03) . ACM, New York, NY, USA, 338–349. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Alexey Gotsman, Hongseok Yang, Carla Ferreira, Mahsa Najafzadeh, and Marc Shapiro. 2016. ’Cause I’m Strong Enough: Reasoning About Consistency Choices in Distributed Systems. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 2016) . ACM, New York, NY, USA, 371–384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Christian Hammer, Julian Dolby, Mandana Vaziri, and Frank Tip. 2008. Dynamic Detection of Atomic-set-serializability Violations. In Proceedings of the 30th International Conference on Software Engineering (ICSE ’08). ACM, New York, NY, USA, 231–240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Jeff Huang, Charles Zhang, and Julian Dolby. 2013. CLAP: Recording Local Executions to Reproduce Concurrency Failures. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’13). ACM, New York, NY, USA, 141–152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jepsen 2018. https://jepsen.io/Google ScholarGoogle Scholar
  26. Sudhir Jorwekar, Alan Fekete, Krithi Ramamritham, and S. Sudarshan. 2007. Automating the Detection of Snapshot Isolation Anomalies. In Proceedings of the 33rd International Conference on Very Large Data Bases, University of Vienna, Austria, September 23-27, 2007 . 1263–1274. http://www.vldb.org/conf/2007/papers/industrial/p1263-jorwekar.pdfGoogle ScholarGoogle Scholar
  27. Gowtham Kaki, Kapil Earanky, KC Sivaramakrishnan, and Suresh Jagannathan. 2018. Safe Replication Through Bounded Concurrency Verification. Proc. ACM Program. Lang. 2, OOPSLA, Article 164 (Oct. 2018), 27 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Charles Killian, James W. Anderson, Ranjit Jhala, and Amin Vahdat. 2007. Life, Death, and the Critical Transition: Finding Liveness Bugs in Systems Code. In Proceedings of the 4th USENIX Conference on Networked Systems Design & Implementation (NSDI’07) . USENIX Association, Berkeley, CA, USA, 18–18. http://dl.acm.org/citation.cfm?id=1973430. 1973448Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Mohsen Lesani, Christian J. Bell, and Adam Chlipala. 2016. Chapar: Certified Causally Consistent Distributed Key-value Stores. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’16) . ACM, New York, NY, USA, 357–370. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Cheng Li, João Leitão, Allen Clement, Nuno Preguiça, Rodrigo Rodrigues, and Viktor Vafeiadis. 2014. Automating the Choice of Consistency Levels in Replicated Systems. In Proceedings of USENIX Annual Technical Conference (USENIX ATC’14) . USENIX Association, Berkeley, CA, USA, 281–292. http://dl.acm.org/citation.cfm?id=2643634.2643664Google ScholarGoogle Scholar
  31. Cheng Li, Daniel Porto, Allen Clement, Johannes Gehrke, Nuno Preguiça, and Rodrigo Rodrigues. 2012. Making Georeplicated Systems Fast As Possible, Consistent when Necessary. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI’12) . USENIX Association, Berkeley, CA, USA, 265–278. http: //dl.acm.org/citation.cfm?id=2387880.2387906Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Shan Lu, Soyeon Park, Chongfeng Hu, Xiao Ma, Weihang Jiang, Zhenmin Li, Raluca A. Popa, and Yuanyuan Zhou. 2007. MUVI: Automatically Inferring Multi-variable Access Correlations and Detecting Related Semantic and Concurrency Bugs. In Proceedings of Twenty-first ACM SIGOPS Symposium on Operating Systems Principles (SOSP ’07). ACM, New York, NY, USA, 103–116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Shan Lu, Joseph Tucek, Feng Qin, and Yuanyuan Zhou. 2006. AVIO: Detecting Atomicity Violations via Access Interleaving Invariants. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XII) . ACM, New York, NY, USA, 37–48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Brandon Lucia, Luis Ceze, and Karin Strauss. 2010. ColorSafe: Architectural Support for Debugging and Dynamically Avoiding Multi-variable Atomicity Violations. In Proceedings of the 37th Annual International Symposium on Computer Architecture (ISCA ’10) . ACM, New York, NY, USA, 222–233. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Nuno Machado, Brandon Lucia, and Luís Rodrigues. 2015. Concurrency Debugging with Differential Schedule Projections. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15). ACM, New York, NY, USA, 586–595. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Bill McCloskey, Feng Zhou, David Gay, and Eric Brewer. 2006. Autolocker: Synchronization Inference for Atomic Sections. In Conference Record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’06). ACM, New York, NY, USA, 346–358. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Kartik Nagar and Suresh Jagannathan. 2018. Automated Detection of Serializability Violations Under Weak Consistency. In 29th International Conference on Concurrency Theory, CONCUR 2018, September 4-7, 2018, Beijing, China. 41:1–41:18. Google ScholarGoogle ScholarCross RefCross Ref
  38. Christos H. Papadimitriou. 1979. The Serializability of Concurrent Database Updates. J. ACM 26, 4 (Oct. 1979), 631–653. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Kia Rahmani, Gowtham Kaki, and Suresh Jagannathan. 2018. Fine-grained Distributed Consistency Guarantees with Effect Orchestration. In Proceedings of the 5th Workshop on the Principles and Practice of Consistency for Distributed Data (PaPoC ’18) . ACM, New York, NY, USA, Article 6, 5 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Kia Rahmani, Kartik Nagar, Benjamin Delaware, and Suresh Jagannatha. 2019. CLOTHO: Directed Test Generation for Weakly Consistent Database Systtems (Extended Version). https://arxiv.org/pdf/1908.05655.pdfGoogle ScholarGoogle Scholar
  41. Sriram Rajamani, G. Ramalingam, Venkatesh Prasad Ranganath, and Kapil Vaswani. 2009. ISOLATOR: Dynamically Ensuring Isolation in Comcurrent Programs. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XIV) . ACM, New York, NY, USA, 181–192. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Dennis Shasha and Philippe Bonnet. 2003. Database Tuning: Principles, Experiments, and Troubleshooting Techniques. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.Google ScholarGoogle Scholar
  43. KC Sivaramakrishnan, Gowtham Kaki, and Suresh Jagannathan. 2015. Declarative Programming over Eventually Consistent Data Stores. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2015) . ACM, New York, NY, USA, 413–424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Ruby Y. Tahboub, Grégory M. Essertel, and Tiark Rompf. 2018. How to Architect a Query Compiler, Revisited. In Proceedings of the 2018 International Conference on Management of Data (SIGMOD ’18) . ACM, New York, NY, USA, 307–322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Raja Vallée-Rai, Phong Co, Etienne Gagnon, Laurie Hendren, Patrick Lam, and Vijay Sundaresan. 1999. Soot - a Java Bytecode Optimization Framework. In Proceedings of the 1999 Conference of the Centre for Advanced Studies on Collaborative Research (CASCON ’99) . IBM Press, 13–. http://dl.acm.org/citation.cfm?id=781995.782008Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Todd Warszawski and Peter Bailis. 2017. ACIDRain: Concurrency-Related Attacks on Database-Backed Web Applications. In Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Conference 2017, Chicago, IL, USA, May 14-19, 2017 . 5–20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Min Xu, Rastislav Bodík, and Mark D. Hill. 2005. A Serializability Violation Detector for Shared-memory Server Programs. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’05). ACM, New York, NY, USA, 1–14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Yang Zhang, Russell Power, Siyuan Zhou, Yair Sovran, Marcos K. Aguilera, and Jinyang Li. 2013. Transaction chains: achieving serializability with low latency in geo-distributed storage systems. In ACM SIGOPS 24th Symposium on Operating Systems Principles, SOSP ’13, Farmington, PA, USA, November 3-6, 2013 . 276–291. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. CLOTHO: directed test generation for weakly consistent database systems

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image Proceedings of the ACM on Programming Languages
      Proceedings of the ACM on Programming Languages  Volume 3, Issue OOPSLA
      October 2019
      2077 pages
      EISSN:2475-1421
      DOI:10.1145/3366395
      Issue’s Table of Contents

      Copyright © 2019 Owner/Author

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 10 October 2019
      Published in pacmpl Volume 3, Issue OOPSLA

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!