skip to main content
research-article
Open Access
Artifacts Evaluated & Reusable

SpaceSearch: a library for building and verifying solver-aided tools

Published:29 August 2017Publication History
Skip Abstract Section

Abstract

Many verification tools build on automated solvers. These tools reduce problems in a specific application domain (e.g., compiler optimization validation) to queries that can be discharged with a highly optimized solver. But the correctness of the reductions themselves is rarely verified in practice, limiting the confidence that the solver's output establishes the desired domain-level property.

This paper presents SpaceSearch, a new library for developing solver-aided tools within a proof assistant. A user builds their solver-aided tool in Coq against the SpaceSearch interface, and the user then verifies that the results provided by the interface are sufficient to establish the tool's desired high-level properties. Once verified, the tool can be extracted to an implementation in a solver-aided language (e.g., Rosette), where SpaceSearch provides an efficient instantiation of the SpaceSearch interface with calls to an underlying SMT solver. This combines the strong correctness guarantees of developing a tool in a proof assistant with the high performance of modern SMT solvers. This paper also introduces new optimizations for such verified solver-aided tools, including parallelization and incrementalization.

We evaluate SpaceSearch by building and verifying two solver-aided tools. The first, SaltShaker, checks that RockSalt's x86 semantics for a given instruction agrees with STOKE's x86 semantics. SaltShaker identified 7 bugs in RockSalt and 1 bug in STOKE. After these systems were patched by their developers, SaltShaker verified the semantics' agreement on 15,255 instruction instantiations in under 2h. The second tool, BGProof, is a verified version of an existing Border Gateway Protocol (BGP) router configuration checker. Like the existing checker, BGProof scales to checking industrial configurations spanning over 240 KLOC, identifying 19 configuration inconsistencies with no false positives. However, the correctness of BGProof has been formally proven, and we found 2 bugs in the unverified implementation. These results demonstrate that SpaceSearch is a practical approach to developing efficient, verified solver-aided tools.

References

  1. Andrew W Appel. 2011. Verified software toolchain. In ESOP. Springer, 1–17.Google ScholarGoogle Scholar
  2. Michael Armand, Germain Faure, Benjamin Grégoire, Chantal Keller, Laurent Théry, and Benjamin Werner. 2011. A Modular Integration of SAT/SMT Solvers to Coq through Proof Witnesses . Springer Berlin Heidelberg, Berlin, Heidelberg, 135–150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Thomas Ball, Nikolaj Bjørner, Aaron Gember, Shachar Itzhaky, Aleksandr Karbyshev, Mooly Sagiv, Michael Schapira, and Asaf Valadarsky. 2014. VeriCon: Towards Verifying Controller Programs in Software-defined Networks. In PLDI. ACM, 282–293. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Clark Barrett, Pascal Fontaine, and Cesare Tinelli. 2016a. The Satisfiability Modulo Theories Library (SMT-LIB). www.SMT-LIB.org. (2016).Google ScholarGoogle Scholar
  5. Clark Barrett, Pascal Fontaine, and Cesare Tinelli. 2016b. The Satisfiability Modulo Theories Library (SMT-LIB). www. smt-lib.org . (2016).Google ScholarGoogle Scholar
  6. Koen Claessen and John Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. In ICFP. Montreal, Canada, 268–279. https://dl.acm.org/citation.cfm?id=1988046Google ScholarGoogle Scholar
  7. Ari Fogel, Stanley Fung, Luis Pedrosa, Meg Walraed-Sullivan, Ramesh Govindan, Ratul Mahajan, and Todd Millstein. 2015. A General Approach to Network Configuration Analysis. In NSDI. USENIX Association, Oakland, CA. https: //dl.acm.org/citation.cfm?id=2789803Google ScholarGoogle Scholar
  8. Stefan Heule, Eric Schkufza, Rahul Sharma, and Alex Aiken. 2016. Stratified Synthesis: Automatically Learning the x86-64 Instruction Set. In PLDI. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Intel. 2015. Intel 64 and IA-32 Architectures Software Developer Manuals, Revision 325462-057US. (Dec. 2015). http: //www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.htmlGoogle ScholarGoogle Scholar
  10. Jinseong Jeon, Xiaokang Qiu, Armando Solar-Lezama, and Jeffrey S. Foster. 2015. Adaptive Concretization for Parallel Program Synthesis. In Computer Aided Verification - 27th International Conference, CAV 2015, San Francisco, CA, USA, July 18-24, 2015, Proceedings, Part II . 377–394. Google ScholarGoogle ScholarCross RefCross Ref
  11. Sudipta Kundu, Zachary Tatlock, and Sorin Lerner. 2009. Proving Optimizations Correct Using Parameterized Program Equivalence. In 2009 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’09). ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Shuvendu K. Lahiri, Shaz Qadeer, and Zvonimir Rakamaric. 2009. Static and Precise Detection of Concurrency Errors in Systems Code Using SMT Solvers. In CAV. Springer Verlag. http://research.microsoft.com/apps/pubs/default.aspx?id= 80360Google ScholarGoogle Scholar
  13. K. Rustan M. Leino. 2008. This is Boogie 2. Technical Report. Microsoft Research. http://research-srv.microsoft.com/en-us/ um/people/leino/papers/krml178.pdfGoogle ScholarGoogle Scholar
  14. K. Rustan M. Leino. 2010. Dafny: An Automatic Program Verifier for Functional Correctness. In LPAR (LPAR’10). SpringerVerlag, 348–370. http://dl.acm.org/citation.cfm?id=1939141.1939161Google ScholarGoogle Scholar
  15. Stéphane Lescuyer and Sylvain Conchon. 2009. Improving Coq Propositional Reasoning Using a Lazy CNF Conversion Scheme. Springer Berlin Heidelberg, Berlin, Heidelberg, 287–303. Google ScholarGoogle ScholarCross RefCross Ref
  16. Guodong Li and Ganesh Gopalakrishnan. 2010. Scalable SMT-based Verification of GPU Kernel Functions. In FSE (FSE’10). ACM, 187–196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Barbara Liskov and Stephen Zilles. 1974. Programming with Abstract Data Types. In Proceedings of the ACM SIGPLAN Symposium on Very High Level Languages . ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Nuno P. Lopes, David Menendez, Santosh Nagarakatte, and John Regehr. 2015. Provably Correct Peephole Optimizations with Alive. In PLDI (PLDI 2015). ACM, 22–32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Filip Marić. 2010. Formal verification of a modern SAT solver by shallow embedding into Isabelle/HOL. TCS 411, 50 (2010), 4333 – 4356. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Greg Morrisett, Gang Tan, Joseph Tassarotti, Jean-Baptiste Tristan, and Edward Gan. 2012. RockSalt: Better, Faster, Stronger SFI for the x86. In PLDI (PLDI ’12). ACM, 395–404. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Duckki Oe, Aaron Stump, Corey Oliver, and Kevin Clancy. 2012. versat: A Verified Modern SAT Solver. Springer Berlin Heidelberg, Berlin, Heidelberg, 363–378. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Pavel Panchekha and Emina Torlak. 2016. Automated Reasoning for Web Page Layout. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2016) . ACM, 181–194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Phitchaya Mangpo Phothilimthana, Tikhon Jelvis, Rohin Shah, Nishant Totla, Sarah Chasins, and Rastislav Bodik. 2014. Chlorophyll: Synthesis-aided Compiler for Low-power Spatial Architectures. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’14) . ACM, 396–407. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Mahmoud Said, Chao Wang, Zijiang Yang, and Karem Sakallah. 2011. Generating Data Race Witnesses by an SMT-based Analysis. In NFM (NFM’11). Springer-Verlag, 313–327. http://dl.acm.org/citation.cfm?id=1986308.1986334 Google ScholarGoogle ScholarCross RefCross Ref
  25. Eric Schkufza, Rahul Sharma, and Alex Aiken. 2013. Stochastic Superoptimization. In ASPLOS (ASPLOS ’13). ACM, 305–316. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Helgi Sigurbjarnarson, James Bornholt, Emina Torlak, and Xi Wang. 2016a. Push-Button Verification of File Systems via Crash Refinement. In OSDI’16. USENIX Association, GA, 1–16. https://www.usenix.org/conference/osdi16/technical-sessions/ presentation/sigurbjarnarsonGoogle ScholarGoogle Scholar
  27. Helgi Sigurbjarnarson, James Bornholt, Emina Torlak, and Xi Wang. 2016b. Push-Button Verification of File Systems via Crash Refinement. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). USENIX Association, GA, 1–16. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/sigurbjarnarsonGoogle ScholarGoogle Scholar
  28. Rohit Singh, Rishabh Singh, Zhilei Xu, Rebecca Krosnick, and Armando Solar-Lezama. 2014. Modular Synthesis of Sketches Using Models. In Verification, Model Checking, and Abstract Interpretation - 15th International Conference, VMCAI 2014, San Diego, CA, USA, January 19-21, 2014, Proceedings . 395–414. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Zachary Tatlock and Sorin Lerner. 2010. Bringing Extensibility to Verified Compilers. In PLDI (PLDI ’10). ACM, 111–121. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Kevin Tew, James Swaine, Matthew Flatt, Robert Bruce Findler, and Peter Dinda. 2014. Distributed Places. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Emina Torlak and Rastislav Bodik. 2013. Growing Solver-aided Languages with Rosette. In Onward! (Onward! 2013). ACM, 135–152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Emina Torlak and Rastislav Bodik. 2014. A Lightweight Symbolic Virtual Machine for Solver-aided Host Languages. In PLDI (PLDI ’14) . ACM, 530–541. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Emina Torlak, Mandana Vaziri, and Julian Dolby. 2010. MemSAT: Checking Axiomatic Specifications of Memory Models. In PLDI (PLDI’10) . ACM, 341–350. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Richard Uhler. 2014. Tutorial 2 - Symbolic Computation. https://github.com/ruhler/smten/blob/master/tutorials/ T2-SymbolicComputation.txt . (2014).Google ScholarGoogle Scholar
  35. Richard Uhler and Nirav Dave. 2014. Smten with Satisfiability-based Search. In OOSPLA (OOPSLA). ACM, 157–176. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Konstantin Weitz, Doug Woos, Emina Torlak, Michael D. Ernst, Arvind Krishnamurthy, and Zachary Tatlock. 2016. Scalable Verification of Border Gateway Protocol Configurations with an SMT Solver. In OOPSLA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. B. Yee, D. Sehr, G. Dardyk, J. B. Chen, R. Muth, T. Ormandy, S. Okasaka, N. Narula, and N. Fullagar. 2009. Native Client: A Sandbox for Portable, Untrusted x86 Native Code. In S&P. 79–93. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. SpaceSearch: a library for building and verifying solver-aided tools

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!