Abstract
Many verification tools build on automated solvers. These tools reduce problems in a specific application domain (e.g., compiler optimization validation) to queries that can be discharged with a highly optimized solver. But the correctness of the reductions themselves is rarely verified in practice, limiting the confidence that the solver's output establishes the desired domain-level property.
This paper presents SpaceSearch, a new library for developing solver-aided tools within a proof assistant. A user builds their solver-aided tool in Coq against the SpaceSearch interface, and the user then verifies that the results provided by the interface are sufficient to establish the tool's desired high-level properties. Once verified, the tool can be extracted to an implementation in a solver-aided language (e.g., Rosette), where SpaceSearch provides an efficient instantiation of the SpaceSearch interface with calls to an underlying SMT solver. This combines the strong correctness guarantees of developing a tool in a proof assistant with the high performance of modern SMT solvers. This paper also introduces new optimizations for such verified solver-aided tools, including parallelization and incrementalization.
We evaluate SpaceSearch by building and verifying two solver-aided tools. The first, SaltShaker, checks that RockSalt's x86 semantics for a given instruction agrees with STOKE's x86 semantics. SaltShaker identified 7 bugs in RockSalt and 1 bug in STOKE. After these systems were patched by their developers, SaltShaker verified the semantics' agreement on 15,255 instruction instantiations in under 2h. The second tool, BGProof, is a verified version of an existing Border Gateway Protocol (BGP) router configuration checker. Like the existing checker, BGProof scales to checking industrial configurations spanning over 240 KLOC, identifying 19 configuration inconsistencies with no false positives. However, the correctness of BGProof has been formally proven, and we found 2 bugs in the unverified implementation. These results demonstrate that SpaceSearch is a practical approach to developing efficient, verified solver-aided tools.
- Andrew W Appel. 2011. Verified software toolchain. In ESOP. Springer, 1–17.Google Scholar
- Michael Armand, Germain Faure, Benjamin Grégoire, Chantal Keller, Laurent Théry, and Benjamin Werner. 2011. A Modular Integration of SAT/SMT Solvers to Coq through Proof Witnesses . Springer Berlin Heidelberg, Berlin, Heidelberg, 135–150. Google Scholar
Digital Library
- Thomas Ball, Nikolaj Bjørner, Aaron Gember, Shachar Itzhaky, Aleksandr Karbyshev, Mooly Sagiv, Michael Schapira, and Asaf Valadarsky. 2014. VeriCon: Towards Verifying Controller Programs in Software-defined Networks. In PLDI. ACM, 282–293. Google Scholar
Digital Library
- Clark Barrett, Pascal Fontaine, and Cesare Tinelli. 2016a. The Satisfiability Modulo Theories Library (SMT-LIB). www.SMT-LIB.org. (2016).Google Scholar
- Clark Barrett, Pascal Fontaine, and Cesare Tinelli. 2016b. The Satisfiability Modulo Theories Library (SMT-LIB). www. smt-lib.org . (2016).Google Scholar
- Koen Claessen and John Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. In ICFP. Montreal, Canada, 268–279. https://dl.acm.org/citation.cfm?id=1988046Google Scholar
- Ari Fogel, Stanley Fung, Luis Pedrosa, Meg Walraed-Sullivan, Ramesh Govindan, Ratul Mahajan, and Todd Millstein. 2015. A General Approach to Network Configuration Analysis. In NSDI. USENIX Association, Oakland, CA. https: //dl.acm.org/citation.cfm?id=2789803Google Scholar
- Stefan Heule, Eric Schkufza, Rahul Sharma, and Alex Aiken. 2016. Stratified Synthesis: Automatically Learning the x86-64 Instruction Set. In PLDI. ACM. Google Scholar
Digital Library
- Intel. 2015. Intel 64 and IA-32 Architectures Software Developer Manuals, Revision 325462-057US. (Dec. 2015). http: //www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.htmlGoogle Scholar
- Jinseong Jeon, Xiaokang Qiu, Armando Solar-Lezama, and Jeffrey S. Foster. 2015. Adaptive Concretization for Parallel Program Synthesis. In Computer Aided Verification - 27th International Conference, CAV 2015, San Francisco, CA, USA, July 18-24, 2015, Proceedings, Part II . 377–394. Google Scholar
Cross Ref
- Sudipta Kundu, Zachary Tatlock, and Sorin Lerner. 2009. Proving Optimizations Correct Using Parameterized Program Equivalence. In 2009 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’09). ACM. Google Scholar
Digital Library
- Shuvendu K. Lahiri, Shaz Qadeer, and Zvonimir Rakamaric. 2009. Static and Precise Detection of Concurrency Errors in Systems Code Using SMT Solvers. In CAV. Springer Verlag. http://research.microsoft.com/apps/pubs/default.aspx?id= 80360Google Scholar
- K. Rustan M. Leino. 2008. This is Boogie 2. Technical Report. Microsoft Research. http://research-srv.microsoft.com/en-us/ um/people/leino/papers/krml178.pdfGoogle Scholar
- K. Rustan M. Leino. 2010. Dafny: An Automatic Program Verifier for Functional Correctness. In LPAR (LPAR’10). SpringerVerlag, 348–370. http://dl.acm.org/citation.cfm?id=1939141.1939161Google Scholar
- Stéphane Lescuyer and Sylvain Conchon. 2009. Improving Coq Propositional Reasoning Using a Lazy CNF Conversion Scheme. Springer Berlin Heidelberg, Berlin, Heidelberg, 287–303. Google Scholar
Cross Ref
- Guodong Li and Ganesh Gopalakrishnan. 2010. Scalable SMT-based Verification of GPU Kernel Functions. In FSE (FSE’10). ACM, 187–196. Google Scholar
Digital Library
- Barbara Liskov and Stephen Zilles. 1974. Programming with Abstract Data Types. In Proceedings of the ACM SIGPLAN Symposium on Very High Level Languages . ACM. Google Scholar
Digital Library
- Nuno P. Lopes, David Menendez, Santosh Nagarakatte, and John Regehr. 2015. Provably Correct Peephole Optimizations with Alive. In PLDI (PLDI 2015). ACM, 22–32. Google Scholar
Digital Library
- Filip Marić. 2010. Formal verification of a modern SAT solver by shallow embedding into Isabelle/HOL. TCS 411, 50 (2010), 4333 – 4356. Google Scholar
Digital Library
- Greg Morrisett, Gang Tan, Joseph Tassarotti, Jean-Baptiste Tristan, and Edward Gan. 2012. RockSalt: Better, Faster, Stronger SFI for the x86. In PLDI (PLDI ’12). ACM, 395–404. Google Scholar
Digital Library
- Duckki Oe, Aaron Stump, Corey Oliver, and Kevin Clancy. 2012. versat: A Verified Modern SAT Solver. Springer Berlin Heidelberg, Berlin, Heidelberg, 363–378. Google Scholar
Digital Library
- Pavel Panchekha and Emina Torlak. 2016. Automated Reasoning for Web Page Layout. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2016) . ACM, 181–194. Google Scholar
Digital Library
- Phitchaya Mangpo Phothilimthana, Tikhon Jelvis, Rohin Shah, Nishant Totla, Sarah Chasins, and Rastislav Bodik. 2014. Chlorophyll: Synthesis-aided Compiler for Low-power Spatial Architectures. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’14) . ACM, 396–407. Google Scholar
Digital Library
- Mahmoud Said, Chao Wang, Zijiang Yang, and Karem Sakallah. 2011. Generating Data Race Witnesses by an SMT-based Analysis. In NFM (NFM’11). Springer-Verlag, 313–327. http://dl.acm.org/citation.cfm?id=1986308.1986334 Google Scholar
Cross Ref
- Eric Schkufza, Rahul Sharma, and Alex Aiken. 2013. Stochastic Superoptimization. In ASPLOS (ASPLOS ’13). ACM, 305–316. Google Scholar
Digital Library
- Helgi Sigurbjarnarson, James Bornholt, Emina Torlak, and Xi Wang. 2016a. Push-Button Verification of File Systems via Crash Refinement. In OSDI’16. USENIX Association, GA, 1–16. https://www.usenix.org/conference/osdi16/technical-sessions/ presentation/sigurbjarnarsonGoogle Scholar
- Helgi Sigurbjarnarson, James Bornholt, Emina Torlak, and Xi Wang. 2016b. Push-Button Verification of File Systems via Crash Refinement. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). USENIX Association, GA, 1–16. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/sigurbjarnarsonGoogle Scholar
- Rohit Singh, Rishabh Singh, Zhilei Xu, Rebecca Krosnick, and Armando Solar-Lezama. 2014. Modular Synthesis of Sketches Using Models. In Verification, Model Checking, and Abstract Interpretation - 15th International Conference, VMCAI 2014, San Diego, CA, USA, January 19-21, 2014, Proceedings . 395–414. Google Scholar
Digital Library
- Zachary Tatlock and Sorin Lerner. 2010. Bringing Extensibility to Verified Compilers. In PLDI (PLDI ’10). ACM, 111–121. Google Scholar
Digital Library
- Kevin Tew, James Swaine, Matthew Flatt, Robert Bruce Findler, and Peter Dinda. 2014. Distributed Places. Google Scholar
Digital Library
- Emina Torlak and Rastislav Bodik. 2013. Growing Solver-aided Languages with Rosette. In Onward! (Onward! 2013). ACM, 135–152. Google Scholar
Digital Library
- Emina Torlak and Rastislav Bodik. 2014. A Lightweight Symbolic Virtual Machine for Solver-aided Host Languages. In PLDI (PLDI ’14) . ACM, 530–541. Google Scholar
Digital Library
- Emina Torlak, Mandana Vaziri, and Julian Dolby. 2010. MemSAT: Checking Axiomatic Specifications of Memory Models. In PLDI (PLDI’10) . ACM, 341–350. Google Scholar
Digital Library
- Richard Uhler. 2014. Tutorial 2 - Symbolic Computation. https://github.com/ruhler/smten/blob/master/tutorials/ T2-SymbolicComputation.txt . (2014).Google Scholar
- Richard Uhler and Nirav Dave. 2014. Smten with Satisfiability-based Search. In OOSPLA (OOPSLA). ACM, 157–176. Google Scholar
Digital Library
- Konstantin Weitz, Doug Woos, Emina Torlak, Michael D. Ernst, Arvind Krishnamurthy, and Zachary Tatlock. 2016. Scalable Verification of Border Gateway Protocol Configurations with an SMT Solver. In OOPSLA. Google Scholar
Digital Library
- B. Yee, D. Sehr, G. Dardyk, J. B. Chen, R. Muth, T. Ormandy, S. Okasaka, N. Narula, and N. Fullagar. 2009. Native Client: A Sandbox for Portable, Untrusted x86 Native Code. In S&P. 79–93. Google Scholar
Digital Library
Index Terms
SpaceSearch: a library for building and verifying solver-aided tools
Recommendations
Scalable verification of border gateway protocol configurations with an SMT solver
OOPSLA 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and ApplicationsInternet Service Providers (ISPs) use the Border Gateway Protocol (BGP) to announce and exchange routes for de- livering packets through the internet. ISPs must carefully configure their BGP routers to ensure traffic is routed reli- ably and securely. ...
Verdi: a framework for implementing and formally verifying distributed systems
PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and ImplementationDistributed systems are difficult to implement correctly because they must handle both concurrency and failures: machines may crash at arbitrary points and networks may reorder, drop, or duplicate packets. Further, their behavior is often too complex ...
Verdi: a framework for implementing and formally verifying distributed systems
PLDI '15Distributed systems are difficult to implement correctly because they must handle both concurrency and failures: machines may crash at arbitrary points and networks may reorder, drop, or duplicate packets. Further, their behavior is often too complex ...






Comments