Abstract
We propose a new conflict-driven program synthesis technique that is capable of learning from past mistakes. Given a spurious program that violates the desired specification, our synthesis algorithm identifies the root cause of the conflict and learns new lemmas that can prevent similar mistakes in the future. Specifically, we introduce the notion of equivalence modulo conflict and show how this idea can be used to learn useful lemmas that allow the synthesizer to prune large parts of the search space. We have implemented a general-purpose CDCL-style program synthesizer called Neo and evaluate it in two different application domains, namely data wrangling in R and functional programming over lists. Our experiments demonstrate the substantial benefits of conflict-driven learning and show that Neo outperforms two state-of-the-art synthesis tools, Morpheus and Deepcoder, that target these respective domains.
Supplemental Material
- Aws Albarghouthi, Sumit Gulwani, and Zachary Kincaid. 2013. Recursive Program Synthesis. In Proc. International Conference on Computer Aided Verification . Springer, 934–950.Google Scholar
Cross Ref
- Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo MK Martin, Mukund Raghothaman, Sanjit A Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. 2013. Syntax-guided synthesis. In Proc. Formal Methods in Computer-Aided Design. IEEE, 1–8.Google Scholar
Cross Ref
- Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2017. Deepcoder: Learning to write programs. In Proc. International Conference on Learning Representations . OpenReview.Google Scholar
- Daniel Le Berre and Anne Parrain. 2010. The Sat4j library, release 2.2. Journal on Satisfiability, Boolean Modeling and Computation (2010), 59–6.Google Scholar
- Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh. 2009. Conflict-driven clause learning SAT solvers. Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications (2009), 131–153.Google Scholar
- Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In Proc. Tools and Algorithms for Construction and Analysis of Systems . Springer, 337–340. Google Scholar
Digital Library
- Yu Feng, Ruben Martins, Osbert Bastani, and Isil Dillig. 2018. Neo. http://utopia-group.github.io/neo/ .Google Scholar
- Yu Feng, Ruben Martins, Jacob Van Geffen, Isil Dillig, and Swarat Chaudhuri. 2017. Component-based synthesis of table consolidation and transformation tasks from examples. In Proc. Conference on Programming Language Design and Implementation . ACM, 422–436. Google Scholar
Digital Library
- Yu Feng, Ruben Martins, Yuepeng Wang, Isil Dillig, and Thomas Reps. 2017. Component-Based Synthesis for Complex APIs. In Proc. Symposium on Principles of Programming Languages . ACM, 599–612. Google Scholar
Digital Library
- John K. Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing Data Structure Transformations from Input-output Examples. In Proc. Conference on Programming Language Design and Implementation . ACM, 229–239. Google Scholar
Digital Library
- Harald Ganzinger, George Hagen, Robert Nieuwenhuis, Albert Oliveras, and Cesare Tinelli. 2004. DPLL( T): Fast Decision Procedures. In Proc. International Conference on Computer Aided Verification . Springer, 175–188.Google Scholar
Cross Ref
- Adrià Gascón, Ashish Tiwari, Brent Carmer, and Umang Mathur. 2017. Look for the Proof to Find the Program: Decorated-Component-Based Program Synthesis. In Proc. International Conference on Computer Aided Verification . Springer, 86–103.Google Scholar
Cross Ref
- Sumit Gulwani. 2011. Automating string processing in spreadsheets using input-output examples. In Proc. Symposium on Principles of Programming Languages . ACM, 317–330. Google Scholar
Digital Library
- Sumit Gulwani, Susmit Jha, Ashish Tiwari, and Ramarathnam Venkatesan. 2011. Synthesis of loop-free programs. In Proc. Conference on Programming Language Design and Implementation . ACM, 62–73. Google Scholar
Digital Library
- Tihomir Gvero, Viktor Kuncak, Ivan Kuraj, and Ruzica Piskac. 2013. Complete completion using types and weights. In Proc. Conference on Programming Language Design and Implementation . ACM, 27–38. Google Scholar
Digital Library
- Susmit Jha, Sumit Gulwani, Sanjit A. Seshia, and Ashish Tiwari. 2010. Oracle-guided component-based program synthesis. In Proc. International Conference on Software Engineering . ACM/IEEE, 215–224. Google Scholar
Digital Library
- David Mandelin, Lin Xu, Rastislav Bodík, and Doug Kimelman. 2005. Jungloid mining: helping to navigate the API jungle. In Proc. Conference on Programming Language Design and Implementation . ACM, 48–61. Google Scholar
Digital Library
- Joao Marques-Silva and Karem A. Sakallah. 1999. GRASP: A Search Algorithm for Propositional Satisfiability. IEEE Trans. Computers 48, 5 (1999), 506–521. Google Scholar
Digital Library
- Aditya Menon, Omer Tamuz, Sumit Gulwani, Butler Lampson, and Adam Kalai. 2013. A machine learning framework for programming by example. In Proc. International Conference on Machine Learning. Proceedings of Machine Learning Research, 187–195. Google Scholar
Digital Library
- Arvind Neelakantan, Quoc V Le, Martin Abadi, Andrew McCallum, and Dario Amodei. 2017. Learning a natural language interface with neural programmer. In Proc. International Conference on Learning Representations . OpenReview.Google Scholar
- Arvind Neelakantan, Quoc V Le, and Ilya Sutskever. 2016. Neural programmer: Inducing latent programs with gradient descent. In Proc. International Conference on Learning Representations . OpenReview.Google Scholar
- Peter-Michael Osera and Steve Zdancewic. 2015. Type-and-exampledirected program synthesis. In Proc. Conference on Programming Language Design and Implementation . ACM, 619–630. Google Scholar
Digital Library
- Nadia Polikarpova, Ivan Kuraj, and Armando Solar-Lezama. 2016. Program synthesis from polymorphic refinement types. Proc. Conference on Programming Language Design and Implementation (2016), 522–538. Google Scholar
Digital Library
- Veselin Raychev, Pavol Bielik, Martin Vechev, and Andreas Krause. 2016. Learning programs from noisy data. In Proc. Symposium on Principles of Programming Languages . ACM, 761–774. Google Scholar
Digital Library
- Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code completion with statistical language models. In Proc. Conference on Programming Language Design and Implementation . ACM, 419–428. Google Scholar
Digital Library
- Rishabh Singh. 2016. Blinkfill: Semi-supervised programming by example for syntactic string transformations. Proceedings of the VLDB Endowment 9, 10 (2016), 816–827. Google Scholar
Digital Library
- Rishabh Singh and Sumit Gulwani. 2016. Transforming spreadsheet data types using examples. In Proc. Symposium on Principles of Programming Languages . ACM, 343–356. Google Scholar
Digital Library
- Calvin Smith and Aws Albarghouthi. 2016. MapReduce program synthesis. In Proc. Conference on Programming Language Design and Implementation . ACM, 326–340. Google Scholar
Digital Library
- Armando Solar-Lezama. 2008. Program synthesis by sketching. University of California, Berkeley.Google Scholar
Digital Library
- Armando Solar-Lezama. 2009. The Sketching Approach to Program Synthesis.. In Proc. Asian Symposium on Programming Languages and Systems . Springer, 4–13. Google Scholar
Digital Library
- Armando Solar-Lezama, Rodric M. Rabbah, Rastislav Bodík, and Kemal Ebcioglu. 2005. Programming by sketching for bit-streaming programs. In Proc. Conference on Programming Language Design and Implementation . ACM, 281–294. Google Scholar
Digital Library
- Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat. 2006. Combinatorial Sketching for Finite Programs. In Proc. International Conference on Architectural Support for Programming Languages and Operating Systems . ACM, 404–415. Google Scholar
Digital Library
- Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems . 3104–3112. Google Scholar
Digital Library
- Ashish Tiwari, Adria Gascón, and Bruno Dutertre. 2015. Program synthesis using dual interpretation. In Proc. International Conference on Automated Deduction . Springer, 482–497.Google Scholar
Cross Ref
- Abhishek Udupa, Arun Raghavan, Jyotirmoy V Deshmukh, Sela MadorHaim, Milo MK Martin, and Rajeev Alur. 2013. TRANSIT: specifying protocols with concolic snippets. Proc. Conference on Programming Language Design and Implementation (2013), 287–296. Google Scholar
Digital Library
- Chenglong Wang, Alvin Cheung, and Rastislav Bodik. 2017. Synthesizing highly expressive SQL queries from input-output examples. In Proc. Conference on Programming Language Design and Implementation . ACM, 452–466. Google Scholar
Digital Library
- Xinyu Wang, Isil Dillig, and Rishabh Singh. 2017. Synthesis of Data Completion Scripts using Finite Tree Automata. In Proc. International Conference on Object-Oriented Programming, Systems, Languages, and Applications . ACM, 62:1–62:26.Google Scholar
Digital Library
- Xinyu Wang, Isil Dillig, and Rishabh Singh. 2018. Program Synthesis using Abstraction Refinement. In Proc. Symposium on Principles of Programming Languages . ACM, 63:1–63:30. Google Scholar
Digital Library
- Navid Yaghmazadeh, Christian Klinger, Isil Dillig, and Swarat Chaudhuri. 2016. Synthesizing transformations on hierarchically structured data. In Proc. Conference on Programming Language Design and Implementation . ACM, 508–521. Google Scholar
Digital Library
- Navid Yaghmazadeh, Yuepeng Wang, Isil Dillig, and Thomas Dillig. 2017. SQLizer: Query Synthesis from Natural Language. In Proc. International Conference on Object-Oriented Programming, Systems, Languages, and Applications . ACM, 63:1–63:26.Google Scholar
Digital Library
- Lintao Zhang, Conor F. Madigan, Matthew W. Moskewicz, and Sharad Malik. 2001. Efficient Conflict Driven Learning in Boolean Satisfiability Solver. In Proc. of International Conference on Computer-Aided Design. IEEE Computer Society, 279–285. Google Scholar
Digital Library
- Sai Zhang and Yuyin Sun. 2013. Automatically synthesizing sql queries from input-output examples. In Proc. International Conference on Automated Software Engineering . IEEE, 224–234. Google Scholar
Digital Library
Index Terms
Program synthesis using conflict-driven learning
Recommendations
Program synthesis using conflict-driven learning
PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and ImplementationWe propose a new conflict-driven program synthesis technique that is capable of learning from past mistakes. Given a spurious program that violates the desired specification, our synthesis algorithm identifies the root cause of the conflict and learns ...
Algorithmic program synthesis: introduction
Program synthesis is a process of producing an executable program from a specification. Algorithmic synthesis produces the program automatically, without an intervention from an expert. While classical compilation falls under the definition of ...
Optimizing synthesis with metasketches
POPL '16Many advanced programming tools---for both end-users and expert developers---rely on program synthesis to automatically generate implementations from high-level specifications. These tools often need to employ tricky, custom-built synthesis algorithms ...







Comments