Abstract
Interactive proofs of theorems often require auxiliary helper lemmas to prove the desired theorem. Existing approaches for automatically synthesizing helper lemmas fall into two broad categories. Some approaches are goal-directed, producing lemmas specifically to help a user make progress from a given proof state, but they have limited expressiveness in terms of the lemmas that can be produced. Other approaches are highly expressive, able to generate arbitrary lemmas from a given grammar, but they are completely undirected and hence not amenable to interactive usage.
In this paper, we develop an approach to lemma synthesis that is both goal-directed and expressive. The key novelty is a technique for reducing lemma synthesis to a data-driven program synthesis problem, whereby examples for synthesis are generated from the current proof state. We also describe a technique to systematically introduce new variables for lemma synthesis, as well as techniques for filtering and ranking candidate lemmas for presentation to the user. We implement these ideas in a tool called lfind, which can be run as a Coq tactic. In an evaluation on four benchmark suites, lfind produces useful lemmas in 68% of the cases where a human prover used a lemma to make progress. In these cases lfind synthesizes a lemma that either enables a fully automated proof of the original goal or that matches the human-provided lemma.
- 1995. Circuits. https://github.com/coq-contribs/circuits
Google Scholar
- 2003. Coq-of-Ocaml. https://github.com/foobar-land/coq-of-ocaml
Google Scholar
- Aws Albarghouthi, Sumit Gulwani, and Zachary Kincaid. 2013. Recursive program synthesis. In International Conference on Computer Aided Verification. 934–950.
Google Scholar
Cross Ref
- Angello Astorga, P Madhusudan, Shambwaditya Saha, Shiyu Wang, and Tao Xie. 2019. Learning stateful preconditions modulo a test generator. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. 775–787.
Google Scholar
Digital Library
- R Aubin. 1976. Mechanising Structural Induction.
Google Scholar
- Kshitij Bansal, Sarah M. Loos, Markus N. Rabe, Christian Szegedy, and Stewart Wilcox. 2019. HOList: An Environment for Machine Learning of Higher-Order Theorem Proving (extended version). CoRR, abs/1904.03241 (2019), arxiv:1904.03241. arxiv:1904.03241
Google Scholar
- Jasmin Christian Blanchette, Sascha Böhme, and Lawrence C. Paulson. 2011. Extending Sledgehammer with SMT Solvers. In Automated Deduction – CADE-23, Nikolaj Bjørner and Viorica Sofronie-Stokkermans (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 116–130. isbn:978-3-642-22438-6
Google Scholar
- Robert S Boyer and J Strother Moore. 1979. A Computational Logic. ACM Monograph Series.
Google Scholar
- Alan Bundy, Andrew Stevens, Frank Van Harmelen, Andrew Ireland, and Alan Smaill. 1993. Rippling: A heuristic for guiding inductive proofs. Artificial intelligence, 62, 2 (1993), 185–253.
Google Scholar
- Jacqueline Castaing. 1985. How to Facilitate the Proof of Theorems by Using the Induction-matching, and by Generalization. In IJCAI.
Google Scholar
- Harsh Raju Chamarthi, Peter C. Dillinger, Matt Kaufmann, and Panagiotis Manolios. 2011. Integrating Testing and Interactive Theorem Proving. In Proceedings 10th International Workshop on the ACL2 Theorem Prover and its Applications, ACL2 2011, Austin, Texas, USA, November 3-4, 2011, David Hardin and Julien Schmaltz (Eds.) (EPTCS, Vol. 70). 4–19. arxiv:1110.4473
Google Scholar
Cross Ref
- Adam Chlipala. 2013. Certified Programming with Dependent Types: A Pragmatic Introduction to the Coq Proof Assistant. MIT Press.
Google Scholar
Digital Library
- Koen Claessen and John Hughes. 2000. QuickCheck: a lightweight tool for random testing of Haskell programs. In 5th ACM SIGPLAN International Conference on Functional Programming (ICFP) (ICFP). ACM, 268–279. http://www.eecs.northwestern.edu/~robby/courses/395-495-2009-fall/quick.pdf
Google Scholar
Digital Library
- Koen Claessen, Moa Johansson, Dan Rosén, and Nicholas Smallbone. 2013. Automating Inductive Proofs Using Theory Exploration. In Automated Deduction - CADE-24 - 24th International Conference on Automated Deduction, Lake Placid, NY, USA, June 9-14, 2013. Proceedings, Maria Paola Bonacina (Ed.) (Lecture Notes in Computer Science, Vol. 7898). Springer, 392–406. isbn:978-3-642-38573-5
Google Scholar
- Koen Claessen, Nicholas Smallbone, and John Hughes. 2010. QuickSpec: Guessing Formal Specifications Using Testing. In [email protected], Gordon Fraser 0001 and Angelo Gargantini (Eds.) (Lecture Notes in Computer Science, Vol. 6143). Springer, 6–21. isbn:978-3-642-13976-5
Google Scholar
- Ł ukasz Czajka and Cezary Kaliszyk. 2018. Hammer for Coq: Automation for Dependent Type Theory. Journal of Automated Reasoning, 61, 1 (2018), 01 Jun, 423–453. issn:1573-0670 https://doi.org/10.1007/s10817-018-9458-4
Google Scholar
Digital Library
- Leonardo de Moura, Soonho Kong, Jeremy Avigad, Floris van Doorn, and Jakob von Raumer. 2015. The Lean Theorem Prover (System Description). In Automated Deduction - CADE-25, Amy P. Felty and Aart Middeldorp (Eds.). Springer International Publishing, Cham. 378–388. isbn:978-3-319-21401-6
Google Scholar
- Lucas Dixon and Jacques Fleuriot. 2003. IsaPlanner: A prototype proof planner in Isabelle. In International Conference on Automated Deduction. 279–283.
Google Scholar
Cross Ref
- Michael D. Ernst. 2000. Dynamically Discovering Likely Program Invariants. University of Washington Department of Computer Science and Engineering. Seattle, Washington.
Google Scholar
- P Ezudheen, Daniel Neider, Deepak D’Souza, Pranav Garg, and P Madhusudan. 2018. Horn-ICE learning for synthesizing invariants and contracts. Proceedings of the ACM on Programming Languages, 2, OOPSLA (2018), 1–25.
Google Scholar
Digital Library
- John K. Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing Data Structure Transformations from Input-Output Examples. SIGPLAN Not., 50, 6 (2015), jun, 229–239. issn:0362-1340 https://doi.org/10.1145/2813885.2737977
Google Scholar
Digital Library
- Jean-Christophe Filliâtre, Hugo Herbelin, Bruno Barras, Bruno Barras, Samuel Boutin, Eduardo Giménez, Samuel Boutin, Gérard Huet, César Muñoz, Cristina Cornes, Cristina Cornes, Judicaël Courant, Judicael Courant, Chetan Murthy, Chetan Murthy, Catherine Parent, Catherine Parent, Christine Paulin-mohring, Christine Paulin-mohring, Amokrane Saibi, Amokrane Saibi, Benjamin Werner, and Benjamin Werner. 1997. The Coq Proof Assistant - Reference Manual Version 6.1.
Google Scholar
- Emily First, Yuriy Brun, and Arjun Guha. 2020. TacTok: Semantics-Aware Proof Synthesis. In Object-oriented Programming, Systems, Languages, and Applications.
Google Scholar
- Jonathan Frankle, Peter-Michael Osera, David Walker, and Steve Zdancewic. 2016. Example-Directed Synthesis: A Type-Theoretic Interpretation. SIGPLAN Not., 51, 1 (2016), jan, 802–815. issn:0362-1340 https://doi.org/10.1145/2914770.2837629
Google Scholar
Digital Library
- Emilio Jesús Gallego Arias, Karl Palmskog, and Vasily Pestun. 2020. SerAPI:Machine-Friendly, Data-Centric Serialization for Coq. https://github.com/ejgallego/coq-serapi
Google Scholar
- Pranav Garg, Christof Löding, P Madhusudan, and Daniel Neider. 2014. ICE: A robust framework for learning invariants. In International Conference on Computer Aided Verification. 69–87.
Google Scholar
Digital Library
- Pranav Garg, Daniel Neider, P. Madhusudan, and Dan Roth. 2016. Learning invariants using decision trees and implication counterexamples. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20 - 22, 2016, Rastislav Bodík and Rupak Majumdar (Eds.). ACM, 499–512. isbn:978-1-4503-3549-2 http://dl.acm.org/citation.cfm?id=2837614
Google Scholar
Digital Library
- Thibault Gauthier, Cezary Kaliszyk, and Josef Urban. 2017. TacticToe: Learning to Reason with HOL4 Tactics. In LPAR-21. 21st International Conference on Logic for Programming, Artificial Intelligence and Reasoning, Thomas Eiter and David Sands (Eds.) (EPiC Series in Computing, Vol. 46). EasyChair, 125–143. issn:2398-7340 https://doi.org/10.29007/ntlb
Google Scholar
Cross Ref
- Jónathan Heras, Ekaterina Komendantskaya, Moa Johansson, and Ewen Maclean. 2013. Proof-Pattern Recognition and Lemma Discovery in ACL2. In Logic for Programming, Artificial Intelligence, and Reasoning - 19th International Conference, LPAR-19, Stellenbosch, South Africa, December 14-19, 2013. Proceedings, Kenneth L. McMillan, Aart Middeldorp, and Andrei Voronkov (Eds.) (Lecture Notes in Computer Science, Vol. 8312). Springer, 389–406. isbn:978-3-642-45220-8
Google Scholar
- Jane Thurmann Hesketh. 1992. Using Middle-Out Reasoning to Guide Inductive Theorem Proving. Ph.D. Dissertation. University of Edinburgh.
Google Scholar
- Daniel Huang, Prafulla Dhariwal, Dawn Song, and Ilya Sutskever. 2019. GamePad: A Learning Environment for Theorem Proving. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum?id=r1xwKoR9Y7
Google Scholar
- B Hummel. 1990. Generation of induction axioms and generalisation.
Google Scholar
- Andrew Ireland and Alan Bundy. 1996. Productive use of failure in inductive proof. In Automated Mathematical Induction. Springer, 79–111.
Google Scholar
- Moa Johansson. 2019. Lemma Discovery for Induction - A Survey. In Intelligent Computer Mathematics - 12th International Conference, CICM 2019, Prague, Czech Republic, July 8-12, 2019, Proceedings, Cezary Kaliszyk, Edwin C. Brady, Andrea Kohlhase, and Claudio Sacerdoti Coen (Eds.) (Lecture Notes in Computer Science, Vol. 11617). Springer, 125–139. isbn:978-3-030-23249-8; 978-3-030-23250-4
Google Scholar
- Moa Johansson, Lucas Dixon, and Alan Bundy. 2010. Dynamic Rippling, Middle-Out Reasoning and Lemma Discovery. In Verification, Induction, Termination Analysis - Festschrift for Christoph Walther on the Occasion of His 60th Birthday, Simon Siegler and Nathan Wasser (Eds.) (Lecture Notes in Computer Science, Vol. 6463). Springer, 102–116. isbn:978-3-642-17171-0
Google Scholar
- Moa Johansson, Lucas Dixon, and Alan Bundy. 2011. Conjecture Synthesis for Inductive Theories. J. Autom. Reason, 47, 3 (2011), 251–289.
Google Scholar
Digital Library
- Moa Johansson, Dan Rosén, Nicholas Smallbone, and Koen Claessen. 2014. Hipster: Integrating Theory Exploration in a Proof Assistant. In Intelligent Computer Mathematics - International Conference, CICM 2014, Coimbra, Portugal, July 7-11, 2014. Proceedings, Stephen M. Watt, James H. Davenport, Alan P. Sexton, Petr Sojka, and Josef Urban (Eds.) (Lecture Notes in Computer Science, Vol. 8543). Springer, 108–122. isbn:978-3-319-08433-6
Google Scholar
- Cezary Kaliszyk and Josef Urban. 2015. HOL(y)Hammer: Online ATP Service for HOL Light. Mathematics in Computer Science, 9, 1 (2015), 01 Mar, 5–22. issn:1661-8289 https://doi.org/10.1007/s11786-014-0182-0
Google Scholar
Cross Ref
- Cezary Kaliszyk and Josef Urban. 2015. MizAR 40 for Mizar 40. Journal of Automated Reasoning, 55, 3 (2015), 01 Oct, 245–256. issn:1573-0670 https://doi.org/10.1007/s10817-015-9330-8
Google Scholar
Digital Library
- Deepak Kapur and Mahadevan Subramaniam. 1996. Lemma Discovery in Automated Induction. In Automated Deduction - CADE-13, 13th International Conference on Automated Deduction, New Brunswick, NJ, USA, July 30 - August 3, 1996, Proceedings, Michael A. McRobbie and John K. Slaney (Eds.) (Lecture Notes in Computer Science, Vol. 1104). Springer, 538–552. isbn:3-540-61511-3
Google Scholar
- Matt Kaufmann and J S. Moore. 1997. An Industrial Strength Theorem Prover for a Logic Based on Common Lisp. IEEE Transactions on Software Engineering, 23, 4 (1997), April, 203–213.
Google Scholar
Digital Library
- Justin Lubin, Nick Collins, Cyrus Omar, and Ravi Chugh. 2020. Program Sketching with Live Bidirectional Evaluation. Proc. ACM Program. Lang., 4, ICFP (2020), Article 109, aug, 29 pages. https://doi.org/10.1145/3408991
Google Scholar
Digital Library
- Anders Miltner, Adrian Trejo Nuñez, Ana Brendel, Swarat Chaudhuri, and Isil Dillig. 2022. Bottom-up Synthesis of Recursive Functional Programs Using Angelic Execution. Proc. ACM Program. Lang., 6, POPL (2022), Article 21, jan, 29 pages. https://doi.org/10.1145/3498682
Google Scholar
Digital Library
- Anders Miltner, Saswat Padhi, David Walker, and Todd Millstein. 2020. Data-driven inference of representation invariants. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation.
Google Scholar
Digital Library
- Omar Montaño-Rivas, Roy L. McCasland, Lucas Dixon, and Alan Bundy. 2012. Scheme-based theorem discovery and concept invention. Expert Syst. Appl, 39, 2 (2012), 1637–1646.
Google Scholar
Digital Library
- Peter-Michael Osera and Steve Zdancewic. 2015. Type-and-example-directed program synthesis. ACM SIGPLAN Notices, 50, 6 (2015), 619–630.
Google Scholar
Digital Library
- Saswat Padhi, Rahul Sharma, and Todd Millstein. 2016. Data-driven precondition inference with learned features. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation. 42–56.
Google Scholar
Digital Library
- Aditya Paliwal, Sarah M. Loos, Markus N. Rabe, Kshitij Bansal, and Christian Szegedy. 2020. Graph Representations for Higher-Order Logic and Theorem Proving. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. AAAI Press, 2967–2974. https://ojs.aaai.org/index.php/AAAI/article/view/5689
Google Scholar
Cross Ref
- Zoe Paraskevopoulou, Aaron Eline, and Leonidas Lampropoulos. 2022. Computing correctly with inductive relations. In Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation. 966–980.
Google Scholar
Digital Library
- Zoe Paraskevopoulou, Cătălin Hriţcu, Maxime Dénès, Leonidas Lampropoulos, and Benjamin Pierce. 2015. Foundational Property-Based Testing. 9236, https://doi.org/10.1007/978-3-319-22102-1_22
Google Scholar
Cross Ref
- Lawrence C. Paulson. 1993. Natural Deduction as Higher-Order Resolution. CoRR, cs.LO/9301104 (1993), arxiv:cs.LO/9301104
Google Scholar
- Andrew Reynolds and Viktor Kuncak. 2015. Induction for SMT Solvers. In VMCAI, Deepak D’Souza, Akash Lal, and Kim Guldstrand Larsen (Eds.) (Lecture Notes in Computer Science, Vol. 8931). Springer, 80–98. isbn:978-3-662-46080-1
Google Scholar
- Alex Sanchez-Stern, Yousef Alhessi, Lawrence K. Saul, and Sorin Lerner. 2020. Generating correctness proofs with neural networks. In Proceedings of the 4th ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, [email protected] 2020, London, UK, June 15, 2020, Koushik Sen and Mayur Naik (Eds.). ACM, 1–10. https://doi.org/10.1145/3394450.3397466
Google Scholar
Digital Library
- Taro Sekiyama, Akifumi Imanishi, and Kohei Suenaga. 2017. Towards Proof Synthesis Guided by Neural Machine Translation for Intuitionistic Propositional Logic. CoRR, abs/1706.06462 (2017), arxiv:1706.06462. arxiv:1706.06462
Google Scholar
- Armando Solar-Lezama. 2009. The Sketching Approach to Program Synthesis. In Programming Languages and Systems, 7th Asian Symposium, APLAS 2009, Seoul, Korea, December 14-16, 2009. Proceedings, Zhenjiang Hu (Ed.) (Lecture Notes in Computer Science, Vol. 5904). Springer, 4–13. isbn:978-3-642-10671-2
Google Scholar
- William Sonnex, Sophia Drossopoulou, and Susan Eisenbach. 2012. Zeno: An automated prover for properties of recursive data structures. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. 407–421.
Google Scholar
Digital Library
- Daniel Whalen. 2016. Holophrasm: a neural Automated Theorem Prover for higher-order logic. arxiv:1608.02644.
Google Scholar
- Kaiyu Yang and Jia Deng. 2019. Learning to Prove Theorems via Interacting with Proof Assistants. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.) (Proceedings of Machine Learning Research, Vol. 97). PMLR, 6984–6994. http://proceedings.mlr.press/v97/yang19a.html
Google Scholar
- Weikun Yang, Grigory Fedyukovich, and Aarti Gupta. 2019. Lemma synthesis for automating induction over algebraic data types. In International Conference on Principles and Practice of Constraint Programming. 600–617.
Google Scholar
Digital Library
- He Zhu, Stephen Magill, and Suresh Jagannathan. 2018. A data-driven CHC solver. ACM SIGPLAN Notices, 53, 4 (2018), 707–721.
Google Scholar
Digital Library
Index Terms
Data-driven lemma synthesis for interactive proofs
Recommendations
Interactive proofs in higher-order concurrent separation logic
POPL '17When using a proof assistant to reason in an embedded logic -- like separation logic -- one cannot benefit from the proof contexts and basic tactics of the proof assistant. This results in proofs that are at a too low level of abstraction because they ...
Interactive proofs in higher-order concurrent separation logic
POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming LanguagesWhen using a proof assistant to reason in an embedded logic -- like separation logic -- one cannot benefit from the proof contexts and basic tactics of the proof assistant. This results in proofs that are at a too low level of abstraction because they ...
Automated lemma synthesis in symbolic-heap separation logic
The symbolic-heap fragment of separation logic has been actively developed and advocated for verifying the memory-safety property of computer programs. At present, one of its biggest challenges is to effectively prove entailments containing inductive ...






Comments