Abstract
This paper addresses the complexity of SAT-based invariant inference, a prominent approach to safety verification. We consider the problem of inferring an inductive invariant of polynomial length given a transition system and a safety property. We analyze the complexity of this problem in a black-box model, called the Hoare-query model, which is general enough to capture algorithms such as IC3/PDR and its variants. An algorithm in this model learns about the system's reachable states by querying the validity of Hoare triples.
We show that in general an algorithm in the Hoare-query model requires an exponential number of queries. Our lower bound is information-theoretic and applies even to computationally unrestricted algorithms, showing that no choice of generalization from the partial information obtained in a polynomial number of Hoare queries can lead to an efficient invariant inference procedure in this class.
We then show, for the first time, that by utilizing rich Hoare queries, as done in PDR, inference can be exponentially more efficient than approaches such as ICE learning, which only utilize inductiveness checks of candidates. We do so by constructing a class of transition systems for which a simple version of PDR with a single frame infers invariants in a polynomial number of queries, whereas every algorithm using only inductiveness checks and counterexamples requires an exponential number of queries.
Our results also shed light on connections and differences with the classical theory of exact concept learning with queries, and imply that learning from counterexamples to induction is harder than classical exact learning from labeled examples. This demonstrates that the convergence rate of Counterexample-Guided Inductive Synthesis depends on the form of counterexamples.
Supplemental Material
- Rajeev Alur, Rastislav Bodík, Eric Dallal, Dana Fisman, Pranav Garg, Garvit Juniwal, Hadas Kress-Gazit, P. Madhusudan, Milo M. K. Martin, Mukund Raghothaman, Shambwaditya Saha, Sanjit A. Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. 2015. Syntax-Guided Synthesis. In Dependable Software Systems Engineering. 1–25.Google Scholar
- Dana Angluin. 1987. Queries and Concept Learning. Machine Learning 2, 4 (1987), 319–342.Google Scholar
Digital Library
- Dana Angluin. 1990. Negative Results for Equivalence Queries. Machine Learning 5 (1990), 121–150.Google Scholar
Digital Library
- Nikolaj Bjørner and Arie Gurfinkel. 2015. Property Directed Polyhedral Abstraction. In Verification, Model Checking, and Abstract Interpretation - 16th International Conference, VMCAI 2015, Mumbai, India, January 12-14, 2015. Proceedings. 263–281. Google Scholar
Digital Library
- Aaron R. Bradley. 2011. SAT-Based Model Checking without Unrolling. In Verification, Model Checking, and Abstract Interpretation - 12th International Conference, VMCAI 2011, Austin, TX, USA, January 23-25, 2011. Proceedings. 70–87. Google Scholar
Cross Ref
- Nader H. Bshouty, Dana Drachsler-Cohen, Martin T. Vechev, and Eran Yahav. 2017. Learning Disjunctions of Predicates. In Proceedings of the 30th Conference on Learning Theory, COLT 2017, Amsterdam, The Netherlands, 7-10 July 2017. 346–369.Google Scholar
- Alessandro Cimatti, Alberto Griggio, Sergio Mover, and Stefano Tonetta. 2014. IC3 Modulo Theories via Implicit Predicate Abstraction. In Tools and Algorithms for the Construction and Analysis of Systems - 20th International Conference, TACAS 2014, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2014, Grenoble, France, April 5-13, 2014. Proceedings. 46–61. Google Scholar
Cross Ref
- Michael Colón, Sriram Sankaranarayanan, and Henny Sipma. 2003. Linear Invariant Generation Using Non-linear Constraint Solving. In Computer Aided Verification, 15th International Conference, CAV 2003, Boulder, CO, USA, July 8-12, 2003, Proceedings. 420–432.Google Scholar
- Patrick Cousot and Radhia Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In Conference Record of the Fourth ACM Symposium on Principles of Programming Languages, Los Angeles, California, USA, January 1977. 238–252. Google Scholar
Digital Library
- Christoph Csallner, Nikolai Tillmann, and Yannis Smaragdakis. 2008. DySy: dynamic symbolic execution for invariant inference. In 30th International Conference on Software Engineering (ICSE 2008), Leipzig, Germany, May 10-18, 2008. 281–290. Google Scholar
Digital Library
- Isil Dillig, Thomas Dillig, Boyang Li, and Kenneth L. McMillan. 2013. Inductive invariant generation via abductive inference. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2013, part of SPLASH 2013, Indianapolis, IN, USA, October 26-31, 2013. 443–456.Google Scholar
- Dana Drachsler-Cohen, Sharon Shoham, and Eran Yahav. 2017. Synthesis with Abstract Examples. In Computer Aided Verification - 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I. 254–278. Google Scholar
Cross Ref
- Niklas Eén, Alan Mishchenko, and Robert K. Brayton. 2011. Efficient implementation of property directed reachability. In International Conference on Formal Methods in Computer-Aided Design, FMCAD ’11, Austin, TX, USA, October 30 -November 02, 2011. 125–134. http://dl.acm.org/citation.cfm?id=2157675Google Scholar
- Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin. 2001. Dynamically Discovering Likely Program Invariants to Support Program Evolution. IEEE Trans. Software Eng. 27, 2 (2001), 99–123. Google Scholar
Digital Library
- P. Ezudheen, Daniel Neider, Deepak D’Souza, Pranav Garg, and P. Madhusudan. 2018. Horn-ICE learning for synthesizing invariants and contracts. PACMPL 2, OOPSLA (2018), 131:1–131:25.Google Scholar
- Grigory Fedyukovich and Rastislav Bodík. 2018. Accelerating Syntax-Guided Invariant Synthesis. In Tools and Algorithms for the Construction and Analysis of Systems - 24th International Conference, TACAS 2018, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2018, Thessaloniki, Greece, April 14-20, 2018, Proceedings, Part I. 251–269. Google Scholar
Cross Ref
- Yotam M. Y. Feldman, Neil Immerman, Mooly Sagiv, and Sharon Shoham. 2020. Complexity and Information in Invariant Inference. Technical Report. arXiv: 1910.12256 https://arxiv.org/abs/1910.12256Google Scholar
- Cormac Flanagan and K. Rustan M. Leino. 2001. Houdini, an Annotation Assistant for ESC/Java. In FME 2001: Formal Methods for Increasing Software Productivity, International Symposium of Formal Methods Europe, Berlin, Germany, March 12-16, 2001, Proceedings. 500–517.Google Scholar
Cross Ref
- Cormac Flanagan and Shaz Qadeer. 2002. Predicate abstraction for software verification. In Conference Record of POPL 2002: The 29th SIGPLAN-SIGACT Symposium on Principles of Programming Languages, Portland, OR, USA, January 16-18, 2002. 191–202. Google Scholar
Digital Library
- Pranav Garg, Christof Löding, P Madhusudan, and Daniel Neider. 2013. ICE: A robust framework for learning invariants. Technical Report. 69–87 pages. http://hdl.handle.net/2142/45973Google Scholar
- Pranav Garg, Christof Löding, P Madhusudan, and Daniel Neider. 2014. ICE: A robust framework for learning invariants. In Computer Aided Verification. Springer, 69–87.Google Scholar
- Pranav Garg, Daniel Neider, P. Madhusudan, and Dan Roth. 2016. Learning invariants using decision trees and implication counterexamples. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20 - 22, 2016. 499–512. Google Scholar
Digital Library
- Sally A. Goldman and Michael J. Kearns. 1995. On the Complexity of Teaching. J. Comput. Syst. Sci. 50, 1 (1995), 20–31. Google Scholar
Digital Library
- Oded Goldreich. 2006. On Promise Problems: A Survey. In Theoretical Computer Science, Essays in Memory of Shimon Even. 254–290.Google Scholar
- Susanne Graf and Hassen Saïdi. 1997. Construction of Abstract State Graphs with PVS. In Computer Aided Verification, 9th International Conference, CAV ’97, Haifa, Israel, June 22-25, 1997, Proceedings. 72–83. Google Scholar
Cross Ref
- Sumit Gulwani. 2012. Synthesis from Examples: Interaction Models and Algorithms. In 14th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2012, Timisoara, Romania, September 26-29, 2012. 8–14. Google Scholar
Digital Library
- Arie Gurfinkel and Alexander Ivrii. 2015. Pushing to the Top. In Formal Methods in Computer-Aided Design, FMCAD 2015, Austin, Texas, USA, September 27-30, 2015. 65–72.Google Scholar
- Armin Haken. 1985. The Intractability of Resolution. Theor. Comput. Sci. 39 (1985), 297–308. Google Scholar
- Lisa Hellerstein, Devorah Kletenik, Linda Sellie, and Rocco A. Servedio. 2012. Tight Bounds on Proper Equivalence Query Learning of DNF. In COLT 2012 - The 25th Annual Conference on Learning Theory, June 25-27, 2012, Edinburgh, Scotland. 31.1–31.18. http://proceedings.mlr.press/v23/hellerstein12/hellerstein12.pdfGoogle Scholar
- Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Kenneth L. McMillan. 2004. Abstractions from proofs. In Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2004, Venice, Italy, January 14-16, 2004. 232–244. Google Scholar
Digital Library
- Krystof Hoder and Nikolaj Bjørner. 2012. Generalized Property Directed Reachability. In Theory and Applications of Satisfiability Testing - SAT 2012 - 15th International Conference, Trento, Italy, June 17-20, 2012. Proceedings. 157–171.Google Scholar
- Bertrand Jeannet, Peter Schrammel, and Sriram Sankaranarayanan. 2014. Abstract acceleration of general linear loops. In The 41st Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’14, San Diego, CA, USA, January 20-21, 2014. 529–540. Google Scholar
Digital Library
- Susmit Jha, Sumit Gulwani, Sanjit A. Seshia, and Ashish Tiwari. 2010. Oracle-guided component-based program synthesis. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1, ICSE 2010, Cape Town, South Africa, 1-8 May 2010. 215–224. Google Scholar
Digital Library
- Susmit Jha and Sanjit A. Seshia. 2017. A theory of formal synthesis via inductive learning. Acta Inf. 54, 7 (2017), 693–726. Google Scholar
Digital Library
- Ranjit Jhala and Kenneth L. McMillan. 2007. Interpolant-Based Transition Relation Approximation. Logical Methods in Computer Science 3, 4 (2007). Google Scholar
Cross Ref
- Aleksandr Karbyshev, Nikolaj Bjørner, Shachar Itzhaky, Noam Rinetzky, and Sharon Shoham. 2017. Property-Directed Inference of Universal Invariants or Proving Their Absence. J. ACM 64, 1 (2017), 7:1–7:33. Google Scholar
Digital Library
- Anvesh Komuravelli, Arie Gurfinkel, and Sagar Chaki. 2014. SMT-Based Model Checking for Recursive Programs. In Computer Aided Verification - 26th International Conference, CAV 2014, Held as Part of the Vienna Summer of Logic, VSL 2014, Vienna, Austria, July 18-22, 2014. Proceedings. 17–34.Google Scholar
- Shuvendu K. Lahiri and Shaz Qadeer. 2009. Complexity and Algorithms for Monomial and Clausal Predicate Abstraction. In Automated Deduction - CADE-22, 22nd International Conference on Automated Deduction, Montreal, Canada, August 2-7, 2009. Proceedings. 214–229.Google Scholar
- Vu Le, Daniel Perelman, Oleksandr Polozov, Mohammad Raza, Abhishek Udupa, and Sumit Gulwani. 2017. Interactive Program Synthesis. CoRR (2017). arXiv: 1703.03539 http://arxiv.org/abs/1703.03539Google Scholar
- Christof Löding, P. Madhusudan, and Daniel Neider. 2016. Abstract Learning Frameworks for Synthesis. In Tools and Algorithms for the Construction and Analysis of Systems - 22nd International Conference, TACAS 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2-8, 2016, Proceedings. 167–185. Google Scholar
Digital Library
- Kenneth L. McMillan. 2003. Interpolation and SAT-Based Model Checking. In Computer Aided Verification, 15th International Conference, CAV 2003, Boulder, CO, USA, July 8-12, 2003, Proceedings. 1–13.Google Scholar
- Kenneth L. McMillan. 2006. Lazy Abstraction with Interpolants. In Computer Aided Verification, 18th International Conference, CAV 2006, Seattle, WA, USA, August 17-20, 2006, Proceedings. 123–136. Google Scholar
Digital Library
- Srinivas Nidhra and Jagruthi Dondeti. 2012. Black Box and White Box Testing Techniques - A Literature Review. International Journal of Embedded Systems and Applications 2 (06 2012), 29–50. Google Scholar
Cross Ref
- Oded Padon, Kenneth L. McMillan, Aurojit Panda, Mooly Sagiv, and Sharon Shoham. 2016. Ivy: safety verification by interactive generalization. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2016, Santa Barbara, CA, USA, June 13-17, 2016. 614–630.Google Scholar
Digital Library
- Knot Pipatsrisawat and Adnan Darwiche. 2011. On the power of clause-learning SAT solvers as resolution engines. Artif. Intell. 175, 2 (2011), 512–525. Google Scholar
Digital Library
- Thomas W. Reps, Shmuel Sagiv, and Greta Yorsh. 2004. Symbolic Implementation of the Best Transformer. In Verification, Model Checking, and Abstract Interpretation, 5th International Conference, VMCAI 2004, Venice, Italy, January 11-13, 2004, Proceedings. 252–266. Google Scholar
Cross Ref
- Robert Robere, Antonina Kolokolova, and Vijay Ganesh. 2018. The Proof Complexity of SMT Solvers. In Computer Aided Verification - 30th International Conference, CAV 2018, Held as Part of the Federated Logic Conference, FloC 2018, Oxford, UK, July 14-17, 2018, Proceedings, Part II. 275–293. Google Scholar
Cross Ref
- Sriram Sankaranarayanan, Swarat Chaudhuri, Franjo Ivancic, and Aarti Gupta. 2008. Dynamic inference of likely data preconditions over predicates by tree learning. In Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2008, Seattle, WA, USA, July 20-24, 2008. 295–306. Google Scholar
Digital Library
- Sriram Sankaranarayanan, Henny B. Sipma, and Zohar Manna. 2004. Constraint-Based Linear-Relations Analysis. In Static Analysis, 11th International Symposium, SAS 2004, Verona, Italy, August 26-28, 2004, Proceedings. 53–68.Google Scholar
- Rahul Sharma and Alex Aiken. 2016. From invariant checking to invariant inference using randomized search. Formal Methods in System Design 48, 3 (2016), 235–256. Google Scholar
Digital Library
- Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken, Percy Liang, and Aditya V. Nori. 2013b. A Data Driven Approach for Algebraic Loop Invariants. In Programming Languages and Systems - 22nd European Symposium on Programming, ESOP 2013, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2013, Rome, Italy, March 16-24, 2013. Proceedings. 574–592. Google Scholar
Digital Library
- Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken, and Aditya V. Nori. 2013a. Verification as Learning Geometric Concepts. In Static Analysis - 20th International Symposium, SAS 2013, Seattle, WA, USA, June 20-22, 2013. Proceedings. 388–411.Google Scholar
- Rahul Sharma, Aditya V. Nori, and Alex Aiken. 2012. Interpolants as Classifiers. In Computer Aided Verification - 24th International Conference, CAV 2012, Berkeley, CA, USA, July 7-13, 2012 Proceedings. 71–87. Google Scholar
Digital Library
- Armando Solar-Lezama, Liviu Tancau, Rastislav Bodík, Sanjit A. Seshia, and Vijay A. Saraswat. 2006. Combinatorial sketching for finite programs. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2006, San Jose, CA, USA, October 21-25, 2006. 404–415. Google Scholar
Digital Library
- Saurabh Srivastava and Sumit Gulwani. 2009. Program verification using templates over predicate abstraction. In Proceedings of the 2009 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2009, Dublin, Ireland, June 15-21, 2009. 223–234.Google Scholar
Digital Library
- Saurabh Srivastava, Sumit Gulwani, and Jeffrey S. Foster. 2013. Template-based program verification and program synthesis. STTT 15, 5-6 (2013), 497–518.Google Scholar
Digital Library
- Aditya V. Thakur, Akash Lal, Junghee Lim, and Thomas W. Reps. 2015. PostHat and All That: Automating Abstract Interpretation. Electr. Notes Theor. Comput. Sci. 311 (2015), 15–32. Google Scholar
Digital Library
- Leslie G. Valiant. 1984. A Theory of the Learnable. Commun. ACM 27, 11 (1984), 1134–1142. Google Scholar
Digital Library
- Yakir Vizel and Orna Grumberg. 2009. Interpolation-sequence based model checking. In Proceedings of 9th International Conference on Formal Methods in Computer-Aided Design, FMCAD 2009, 15-18 November 2009, Austin, Texas, USA. 1–8. Google Scholar
Cross Ref
- Yakir Vizel, Orna Grumberg, and Sharon Shoham. 2013. Intertwined Forward-Backward Reachability Analysis Using Interpolants. In Tools and Algorithms for the Construction and Analysis of Systems - 19th International Conference, TACAS 2013, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2013, Rome, Italy, March 16-24, 2013. Proceedings. 308–323. Google Scholar
Digital Library
- Yakir Vizel and Arie Gurfinkel. 2014. Interpolating Property Directed Reachability. In Computer Aided Verification - 26th International Conference, CAV 2014, Held as Part of the Vienna Summer of Logic, VSL 2014, Vienna, Austria, July 18-22, 2014. Proceedings. 260–276. Google Scholar
Digital Library
- Yakir Vizel, Arie Gurfinkel, Sharon Shoham, and Sharad Malik. 2017. IC3 - Flipping the E in ICE. In Verification, Model Checking, and Abstract Interpretation - 18th International Conference, VMCAI 2017, Paris, France, January 15-17, 2017, Proceedings. 521–538.Google Scholar
Index Terms
Complexity and information in invariant inference
Recommendations
Property-directed reachability as abstract interpretation in the monotone theory
Inferring inductive invariants is one of the main challenges of formal verification. The theory of abstract interpretation provides a rich framework to devise invariant inference algorithms. One of the latest breakthroughs in invariant inference is ...
Learning the boundary of inductive invariants
We study the complexity of invariant inference and its connections to exact concept learning. We define a condition on invariants and their geometry, called the fence condition, which permits applying theoretical results from exact concept learning to ...
Verification and refutation of C programs based on k-induction and invariant inference
AbstractDepthK is a source-to-source transformation tool that employs bounded model checking (BMC) to verify and falsify safety properties in single- and multi-threaded C programs, without manual annotation of loop invariants. Here, we describe and ...






Comments