Abstract
The last decade has sparked several valiant efforts in deductive verification of distributed agreement protocols such as consensus and leader election. Oddly, there have been far fewer verification efforts that go beyond the core protocols and target applications that are built on top of agreement protocols. This is unfortunate, as agreement-based distributed services such as data stores, locks, and ledgers are ubiquitous and potentially permit modular, scalable verification approaches that mimic their modular design. We address this need for verification of distributed agreement-based systems through our novel modeling and verification framework, QuickSilver, that is not only modular, but also fully automated. The key enabling feature of QuickSilver is our encoding of abstractions of verified agreement protocols that facilitates modular, decidable, and scalable automated verification. We demonstrate the potential of QuickSilver by modeling and efficiently verifying a series of tricky case studies, adapted from real-world applications, such as a data store, a lock service, a surveillance system, a pathfinding algorithm for mobile robots, and more.
Supplemental Material
- Rajeev Alur, Milo Martin, Mukund Raghothaman, Christos Stergiou, Stavros Tripakis, and Abhishek Udupa. 2014. Synthesizing Finite-State Protocols from Scenarios and Requirements. In Hardware and Software: Verification and Testing, Eran Yahav (Ed.). Springer International Publishing, Cham. 75–91. isbn:978-3-319-13338-6Google Scholar
- Rajeev Alur, Mukund Raghothaman, Christos Stergiou, Stavros Tripakis, and Abhishek Udupa. 2015. Automatic Completion of Distributed Protocols with Symmetry. In Computer Aided Verification, Daniel Kroening and Corina S. Păsăreanu (Eds.). Springer International Publishing, Cham. 395–412. isbn:978-3-319-21668-3Google Scholar
- Rajeev Alur and Stavros Tripakis. 2017. Automatic Synthesis of Distributed Protocols. SIGACT News, 48, 1 (2017), March, 55–90. issn:0163-5700 https://doi.org/10.1145/3061640.3061652 Google Scholar
Digital Library
- Benjamin Aminof, Tomer Kotek, Sasha Rubin, Francesco Spegni, and Helmut Veith. 2018. Parameterized model checking of rendezvous systems. Distributed Computing, 31, 3 (2018), 187–222. https://doi.org/10.1007/s00446-017-0302-6 Google Scholar
Digital Library
- Zachary Amsden, Ramnik Arora, Shehar Bano, Mathieu Baudet, Sam Blackshear, Abhay Bothra, George Cabrera andChristian Catalini, Konstantinos Chalkias, Evan Cheng, Avery Ching, Andrey Chursin, George Danezis andGerardo Di Giacomo, David L. Dill, Hui Ding, Nick Doudchenko, Victor Gao, Zhenhuan Gao, François Garillot, Michael Gorven, Philip Hayes, J. Mark Hou, Yuxuan Hu, Kevin Hurley, Kevin Lewi, Chunqi Li, Zekun Li, Dahlia Malkhi andSonia Margulis, Ben Maurer, Payman Mohassel, Ladi de Naurois, Valeria Nikolaenko, Todd Nowacki, Oleksandr Orlov andDmitri Perelman, Alistair Pott, Brett Proctor, Shaz Qadeer, Rain, Dario Russi, Bryan Schwab, Stephane Sezer, Alberto Sonnino, Herman Venter, Lei Wei, Nils Wernerfelt, Brandon Williams, Qinfan Wu, Xifan Yan, Tim Zakian, and Runtian Zhou. 2020. The Libra Blockchain. https://developers.libra.org/docs/assets/papers/the-libra-blockchain/2020-05-26.pdfGoogle Scholar
- Kristoffer Just Arndal Andersen and Ilya Sergey. 2019. Distributed Protocol Combinators. In Practical Aspects of Declarative Languages, José Júlio Alferes and Moa Johansson (Eds.). Springer International Publishing, Cham. 169–186. isbn:978-3-030-05998-9Google Scholar
- Krzysztof R. Apt and Dexter C. Kozen. 1986. Limits for automatic verification of finite-state concurrent systems. Inform. Process. Lett., 22, 6 (1986), 307–309. issn:0020-0190 https://doi.org/10.1016/0020-0190(86)90071-2 Google Scholar
Digital Library
- A. Arghavani, E. Ahmadi, and A. T. Haghighat. 2011. Improved bully election algorithm in distributed systems. In ICIMU 2011 : Proceedings of the 5th international Conference on Information Technology Multimedia. 1–6. https://doi.org/10.1109/ICIMU.2011.6122724 Google Scholar
Cross Ref
- Atomix. 2021. Atomix. https://atomix.io/docs/latest/user-manual/primitives/AtomicValue/Google Scholar
- Simon Auß erlechner, Swen Jacobs, and Ayrat Khalimov. 2016. Tight Cutoffs for Guarded Protocols with Fairness. In Verification, Model Checking, and Abstract Interpretation - 17th International Conference, VMCAI 2016, St. Petersburg, FL, USA, January 17-19, 2016. Proceedings, Barbara Jobstmann and K. Rustan M. Leino (Eds.) (Lecture Notes in Computer Science, Vol. 9583). Springer, 476–494. https://doi.org/10.1007/978-3-662-49122-5_23 Google Scholar
Digital Library
- Roderick Bloem, Nicolas Braud-Santoni, and Swen Jacobs. 2016. Synthesis of Self-Stabilising and Byzantine-Resilient Distributed Systems. In Computer Aided Verification, Swarat Chaudhuri and Azadeh Farzan (Eds.). Springer International Publishing, Cham. 157–176. isbn:978-3-319-41528-4Google Scholar
- Mike Burrows. 2006. The Chubby Lock Service for Loosely-Coupled Distributed Systems. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI ’06). USENIX Association, USA. 335–350. isbn:1931971471Google Scholar
Digital Library
- Davide Canepa and Maria Gradinariu Potop-Butucaru. 2007. Stabilizing Flocking Via Leader Election in Robot Networks. In Stabilization, Safety, and Security of Distributed Systems, Toshimitsu Masuzawa and Sébastien Tixeuil (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 52–66. isbn:978-3-540-76627-8Google Scholar
- Saksham Chand, Yanhong A. Liu, and Scott D. Stoller. 2016. Formal Verification of Multi-Paxos for Distributed Consensus. In FM 2016: Formal Methods, John Fitzgerald, Constance Heitmeyer, Stefania Gnesi, and Anna Philippou (Eds.). Springer International Publishing, Cham. 119–136. isbn:978-3-319-48989-6Google Scholar
- Tushar D Chandra, Robert Griesemer, and Joshua Redstone. 2007. Paxos Made Live: an Engineering Perspective. In Proceedings of the twenty-sixth annual ACM symposium on Principles of distributed computing. 398–407.Google Scholar
Digital Library
- Che-Cheng Chang and Jichiang Tsai. 2016. Distributed collaborative surveillance system based on leader election protocols. IET Wireless Sensor Systems, 6, 6 (2016), 198–205. https://doi.org/10.1049/iet-wss.2015.0030 Google Scholar
Cross Ref
- Bernadette Charron-Bost and André Schiper. 2009. The Heard-of Model: Computing in Distributed Systems with Benign Faults. Distributed Computing, 22, 1 (2009), 49–71. https://doi.org/10.1007/s00446-009-0084-6 Google Scholar
Digital Library
- Denis Cousineau, Damien Doligez, Leslie Lamport, Stephan Merz, Daniel Ricketts, and Hernán Vanzetto. 2012. TLA+ Proofs. In International Symposium on Formal Methods. 147–154.Google Scholar
Cross Ref
- Andrei Damian, Cezara Dragoi, Alexandru Militaru, and Josef Widder. 2019. Communication-closed Asynchronous Protocols. In International Conference on Computer Aided Verification.Google Scholar
- Werner Damm and Bernd Finkbeiner. 2014. Automatic Compositional Synthesis of Distributed Systems. In International Symposium on Formal Methods. 179–193.Google Scholar
- Giorgio Delzanno, Jean-François Raskin, and Laurent Van Begin. 2002. Towards the Automated Verification of Multithreaded Java Programs. In TACAS (Lecture Notes in Computer Science, Vol. 2280). Springer, 173–187.Google Scholar
Cross Ref
- Ankush Desai, Indranil Saha, Jianqiao Yang, Shaz Qadeer, and Sanjit A. Seshia. 2017. DRONA: A Framework for Safe Distributed Mobile Robotics. In Proceedings of the 8th International Conference on Cyber-Physical Systems (ICCPS ’17). ACM, 239–248. isbn:978-1-4503-4965-9Google Scholar
- Ryan Doenges, James R Wilcox, Doug Woos, Zachary Tatlock, and Karl Palmskog. 2017. Verification of Implementations of Distributed Systems Under Churn.Google Scholar
- Cezara Drăgoi, Thomas A Henzinger, Helmut Veith, Josef Widder, and Damien Zufferey. 2014. A Logic-based Framework for Verifying Consensus Algorithms. In International Conference on Verification, Model Checking, and Abstract Interpretation. 161–181.Google Scholar
Digital Library
- Cezara Drăgoi, Thomas A. Henzinger, and Damien Zufferey. 2016. PSync: A Partially Synchronous Language for Fault-Tolerant Distributed Algorithms. SIGPLAN Not., 51, 1 (2016), Jan., 400–415. issn:0362-1340 https://doi.org/10.1145/2914770.2837650 Google Scholar
Digital Library
- E. Allen Emerson and Vineet Kahlon. 2003. Exact and Efficient Verification of Parameterized Cache CoherenceProtocols. In CHARME (Lecture Notes in Computer Science, Vol. 2860). Springer, 247–262.Google Scholar
- E. Allen Emerson and Vineet Kahlon. 2003. Model Checking Guarded Protocols. In 18th IEEE Symposium on Logic in Computer Science (LICS 2003), 22-25 June 2003, Ottawa, Canada, Proceedings. IEEE Computer Society, 361–370.Google Scholar
- E. Allen Emerson and A Prasad Sistla. 1996. Symmetry and Model Checking. Formal methods in system design, 9, 1-2 (1996), 105–131.Google Scholar
- E. Allen Emerson and Thomas Wahl. 2003. On Combining Symmetry Reduction and Symbolic Representation for Efficient Model Checking. In Advanced Research Working Conference on Correct Hardware Design and Verification Methods. 216–230.Google Scholar
- Javier Esparza, Alain Finkel, and Richard Mayr. 1999. On the Verification of Broadcast Protocols. In 14th Annual IEEE Symposium on Logic in Computer Science, Trento, Italy, July 2-5, 1999. IEEE Computer Society, 352–359. https://doi.org/10.1109/LICS.1999.782630 Google Scholar
- Yotam M. Y. Feldman, James R. Wilcox, Sharon Shoham, and Mooly Sagiv. 2019. Inferring Inductive Invariants from Phase Structures. In Computer Aided Verification, Isil Dillig and Serdar Tasiran (Eds.). Springer International Publishing, Cham. 405–425. isbn:978-3-030-25543-5Google Scholar
- Hector Garcia-Molina. 1982. Elections in a distributed computing system. IEEE Computer Architecture Letters, 31, 01 (1982), 48–59. https://doi.org/10.1109/TC.1982.1675885 Google Scholar
Digital Library
- Álvaro García-Pérez, Alexey Gotsman, Yuri Meshman, and Ilya Sergey. 2018. Paxos Consensus, Deconstructed and Abstracted. In Programming Languages and Systems, Amal Ahmed (Ed.). Springer International Publishing, Cham. 912–939.Google Scholar
- Steven M. German and A. Prasad Sistla. 1992. Reasoning about Systems with Many Processes. J. ACM, 39, 3 (1992), July, 675–735. issn:0004-5411 https://doi.org/10.1145/146637.146681 Google Scholar
Digital Library
- Jeremiah Griffin, Mohsen Lesani, Narges Shadab, and Xizhe Yin. 2020. TLC: Temporal Logic of Distributed Components. Proc. ACM Program. Lang., 4, ICFP (2020), Article 123, Aug., 30 pages. https://doi.org/10.1145/3409005 Google Scholar
Digital Library
- Chris Hawblitzel, Jon Howell, Manos Kapritsos, Jacob R. Lorch, Bryan Parno, Michael L. Roberts, Srinath Setty, and Brian Zill. 2015. IronFleet: Proving Practical Distributed Systems Correct. In Proceedings of the 25th Symposium on Operating Systems Principles (SOSP ’15). Association for Computing Machinery, New York, NY, USA. 1–17. isbn:9781450338349 https://doi.org/10.1145/2815400.2815428 Google Scholar
Digital Library
- Hyperledger. 2021. The Hyperledger Project. https://www.hyperledger.org/Google Scholar
- C Norris Ip and David L Dill. 1996. Better Verification Through Symmetry. Formal methods in system design, 9, 1-2 (1996), 41–75.Google Scholar
- Nouraldin Jaber, Swen Jacobs, Christopher Wagner, Milind Kulkarni, and Roopsha Samanta. 2020. Parameterized Verification of Systems with Global Synchronization and Guards. In Computer Aided Verification, Shuvendu K. Lahiri and Chao Wang (Eds.). Springer International Publishing, Cham. 299–323. isbn:978-3-030-53288-8Google Scholar
- Nouraldin Jaber, Christopher Wagner, Swen Jacobs, Milind Kulkarni, and Roopsha Samanta. 2020. Parameterized Reasoning for Distributed Systems with Consensus. CoRR, abs/2004.04613 (2020), arXiv:2004.04613. arxiv:2004.04613Google Scholar
- Swen Jacobs and Mouhammad Sakr. 2018. Analyzing Guarded Protocols: Better Cutoffs, More Systems, More Expressivity. In Verification, Model Checking, and Abstract Interpretation - 19th International Conference, VMCAI 2018, Los Angeles, CA, USA, January 7-9, 2018, Proceedings, Isil Dillig and Jens Palsberg (Eds.) (Lecture Notes in Computer Science, Vol. 10747). Springer, 247–268. https://doi.org/10.1007/978-3-319-73721-8_12 Google Scholar
- Bernhard Kragl, Constantin Enea, Thomas A. Henzinger, Suha Orhun Mutluergil, and Shaz Qadeer. 2020. Inductive Sequentialization of Asynchronous Programs. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2020). Association for Computing Machinery, New York, NY, USA. 227–242. isbn:9781450376136 https://doi.org/10.1145/3385412.3385980 Google Scholar
Digital Library
- Morten Krogh-Jespersen, Amin Timany, Marit Edna Ohlenbusch, Simon Oddershede Gregersen, and Lars Birkedal. 2020. Aneris: A Mechanised Logic for Modular Reasoning about Distributed Systems. In Programming Languages and Systems, Peter Müller (Ed.). Springer International Publishing, Cham. 336–365. isbn:978-3-030-44914-8Google Scholar
- Leslie Lamport. 1998. The Part-Time Parliament. ACM Trans. Comput. Syst., 16, 2 (1998), May, 133–169. issn:0734-2071 https://doi.org/10.1145/279227.279229 Google Scholar
Digital Library
- Leslie Lamport. 2002. Specifying Systems: the TLA+ Language and Tools for Hardware and Software Engineers. Addison-Wesley Longman Publishing Co., Inc..Google Scholar
Digital Library
- Leslie Lamport. 2006. Fast paxos. Distributed Computing, 19, 2 (2006), 79–103. https://doi.org/10.1007/s00446-006-0005-x Google Scholar
Digital Library
- Richard J. Lipton. 1975. Reduction: A Method of Proving Properties of Parallel Programs. Commun. ACM, 18, 12 (1975), Dec., 717–721. issn:0001-0782 https://doi.org/10.1145/361227.361234 Google Scholar
Digital Library
- Yanhong A. Liu, Scott D. Stoller, Bo Lin, and Michael Gorbovitski. 2012. From Clarity to Efficiency for Distributed Algorithms. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA ’12). Association for Computing Machinery, New York, NY, USA. 395–410. isbn:9781450315616 https://doi.org/10.1145/2384616.2384645 Google Scholar
Digital Library
- Nancy A. Lynch. 1996. Distributed Algorithms. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. isbn:1558603484Google Scholar
Digital Library
- Yanhua Mao, Flavio P. Junqueira, and Keith Marzullo. 2008. Mencius: Building Efficient Replicated State Machines for WANs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI’08). USENIX Association, USA. 369–384. https://doi.org/10.5555/1855741.1855767Google Scholar
Digital Library
- Ognjen Marić, Christoph Sprenger, and David Basin. 2017. Cutoff Bounds for Consensus Algorithms. In International Conference on Computer Aided Verification. 217–237.Google Scholar
- NASA. 2021. NASA - Small Aircraft Transportation System. https://www.nasa.gov/centers/langley/news/factsheets/SATS.htmlGoogle Scholar
- Diego Ongaro and John K Ousterhout. 2014. In Search of an Understandable Consensus Algorithm.. In USENIX Annual Technical Conference. 305–319.Google Scholar
Digital Library
- Oded Padon, Jochen Hoenicke, Giuliano Losa, Andreas Podelski, Mooly Sagiv, and Sharon Shoham. 2017. Reducing Liveness to Safety in First-Order Logic. Proc. ACM Program. Lang., 2, POPL (2017), Article 26, Dec., 33 pages. https://doi.org/10.1145/3158114 Google Scholar
Digital Library
- Oded Padon, Giuliano Losa, Mooly Sagiv, and Sharon Shoham. 2017. Paxos Made EPR: Decidable Reasoning about Distributed Protocols. Proc. ACM Program. Lang., 1, OOPSLA (2017), Article 108, Oct., 31 pages. https://doi.org/10.1145/3140568 Google Scholar
Digital Library
- Oded Padon, Kenneth L. McMillan, Aurojit Panda, Mooly Sagiv, and Sharon Shoham. 2016. Ivy: Safety Verification by Interactive Generalization. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’16). Association for Computing Machinery, New York, NY, USA. 614–630. isbn:9781450342612 https://doi.org/10.1145/2908080.2908118 Google Scholar
Digital Library
- Ruzica Piskac, Leonardo de Moura, and Nikolaj Bjørner. 2010. Deciding Effectively Propositional Logic Using DPLL and Substitution Sets. Journal of Automated Reasoning, 44, 4 (2010), 401–424.Google Scholar
Digital Library
- QuickSilver. 2021. QuickSilver Implementation. https://doi.org/10.5281/zenodo.5501650 Google Scholar
Digital Library
- Vincent Rahli. 2012. Interfacing with Proof Assistants for Domain Specific Programming Using EventML.Google Scholar
- RedisRaft. 2021. RedisRaft. https://github.com/RedisLabs/redisraft/Google Scholar
- Sylvain Schmitz and Philippe Schnoebelen. 2013. The Power of Well-Structured Systems. In CONCUR 2013, Pedro R. D’Argenio and Hernán C. Melgratti (Eds.) (Lecture Notes in Computer Science, Vol. 8052). Springer, 5–24. https://doi.org/10.1007/978-3-642-40184-8_2 Google Scholar
Digital Library
- Ilya Sergey, James R. Wilcox, and Zachary Tatlock. 2017. Programming and Proving with Distributed Protocols. Proc. ACM Program. Lang., 2, POPL (2017), Article 28, Dec., 30 pages. https://doi.org/10.1145/3158116 Google Scholar
Digital Library
- Ichiro Suzuki. 1988. Proving Properties of a Ring of Finite-State Machines. Inf. Process. Lett., 28, 4 (1988), July, 213–214. issn:0020-0190 https://doi.org/10.1016/0020-0190(88)90211-6 Google Scholar
Digital Library
- Marcelo Taube, Giuliano Losa, Kenneth L. McMillan, Oded Padon, Mooly Sagiv, Sharon Shoham, James R. Wilcox, and Doug Woos. 2018. Modularity for Decidability of Deductive Verification with Applications to Distributed Systems. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2018). Association for Computing Machinery, New York, NY, USA. 662–677. isbn:9781450356985 https://doi.org/10.1145/3192366.3192414 Google Scholar
Digital Library
- Klaus v. Gleissenthall, Rami Gökhan Kıcı, Alexander Bakst, Deian Stefan, and Ranjit Jhala. 2019. Pretend Synchrony: Synchronous Verification of Asynchronous Distributed Programs. Proc. ACM Program. Lang., 3, POPL (2019), Article 59, Jan., 30 pages. https://doi.org/10.1145/3290372 Google Scholar
Digital Library
- Thomas Wahl. 2007. Adaptive Symmetry Reduction. In International Conference on Computer Aided Verification. 393–405.Google Scholar
- James R. Wilcox, Ilya Sergey, and Zachary Tatlock. 2017. Programming Language Abstractions for Modularly Verified Distributed Systems. In 2nd Summit on Advances in Programming Languages (SNAPL 2017), Benjamin S. Lerner, Rastislav Bodík, and Shriram Krishnamurthi (Eds.) (Leibniz International Proceedings in Informatics (LIPIcs), Vol. 71). Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany. 19:1–19:12. isbn:978-3-95977-032-3 issn:1868-8969Google Scholar
- James R. Wilcox, Doug Woos, Pavel Panchekha, Zachary Tatlock, Xi Wang, Michael D. Ernst, and Thomas Anderson. 2015. Verdi: A Framework for Implementing and Formally Verifying Distributed Systems. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15). Association for Computing Machinery, New York, NY, USA. 357–368. isbn:9781450334686 https://doi.org/10.1145/2737924.2737958 Google Scholar
Digital Library
- Doug Woos, James R. Wilcox, Steve Anton, Zachary Tatlock, Michael D. Ernst, and Thomas Anderson. 2016. Planning for Change in a Formal Verification of the Raft Consensus Protocol. In Proceedings of the 5th ACM SIGPLAN Conference on Certified Programs and Proofs (CPP 2016). Association for Computing Machinery, New York, NY, USA. 154–165. isbn:9781450341271 https://doi.org/10.1145/2854065.2854081 Google Scholar
Digital Library
- Junfeng Yang, Tisheng Chen, Ming Wu, Zhilei Xu, Xuezheng Liu, Haoxiang Lin, Mao Yang, Fan Long, Lintao Zhang, and Lidong Zhou. 2009. MODIST: Transparent Model Checking of Unmodified Distributed Systems. In Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation (NSDI’09). USENIX Association, USA. 213–228.Google Scholar
Digital Library
Index Terms
QuickSilver: modeling and parameterized verification for distributed agreement-based systems
Recommendations
Parameterized verification of transactional memories
PLDI '10We describe an automatic verification method to check whether transactional memories ensure strict serializability a key property assumed of the transactional interface. Our main contribution is a technique for effectively verifying parameterized ...
Parameterized verification of transactional memories
PLDI '10: Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Design and ImplementationWe describe an automatic verification method to check whether transactional memories ensure strict serializability a key property assumed of the transactional interface. Our main contribution is a technique for effectively verifying parameterized ...
An automata-theoretic approach to modular model checking
In modular verification the specification of a module consists of two part. One part describes the guaranteed behavior of the module. The other part describes the assumed behavior of the system in which the module is interacting. This is called the ...






Comments