skip to main content
research-article
Public Access

Program synthesis using conflict-driven learning

Published:11 June 2018Publication History
Skip Abstract Section

Abstract

We propose a new conflict-driven program synthesis technique that is capable of learning from past mistakes. Given a spurious program that violates the desired specification, our synthesis algorithm identifies the root cause of the conflict and learns new lemmas that can prevent similar mistakes in the future. Specifically, we introduce the notion of equivalence modulo conflict and show how this idea can be used to learn useful lemmas that allow the synthesizer to prune large parts of the search space. We have implemented a general-purpose CDCL-style program synthesizer called Neo and evaluate it in two different application domains, namely data wrangling in R and functional programming over lists. Our experiments demonstrate the substantial benefits of conflict-driven learning and show that Neo outperforms two state-of-the-art synthesis tools, Morpheus and Deepcoder, that target these respective domains.

Skip Supplemental Material Section

Supplemental Material

p420-feng.webm

References

  1. Aws Albarghouthi, Sumit Gulwani, and Zachary Kincaid. 2013. Recursive Program Synthesis. In Proc. International Conference on Computer Aided Verification . Springer, 934–950.Google ScholarGoogle ScholarCross RefCross Ref
  2. Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo MK Martin, Mukund Raghothaman, Sanjit A Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. 2013. Syntax-guided synthesis. In Proc. Formal Methods in Computer-Aided Design. IEEE, 1–8.Google ScholarGoogle ScholarCross RefCross Ref
  3. Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2017. Deepcoder: Learning to write programs. In Proc. International Conference on Learning Representations . OpenReview.Google ScholarGoogle Scholar
  4. Daniel Le Berre and Anne Parrain. 2010. The Sat4j library, release 2.2. Journal on Satisfiability, Boolean Modeling and Computation (2010), 59–6.Google ScholarGoogle Scholar
  5. Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh. 2009. Conflict-driven clause learning SAT solvers. Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications (2009), 131–153.Google ScholarGoogle Scholar
  6. Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In Proc. Tools and Algorithms for Construction and Analysis of Systems . Springer, 337–340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Yu Feng, Ruben Martins, Osbert Bastani, and Isil Dillig. 2018. Neo. http://utopia-group.github.io/neo/ .Google ScholarGoogle Scholar
  8. Yu Feng, Ruben Martins, Jacob Van Geffen, Isil Dillig, and Swarat Chaudhuri. 2017. Component-based synthesis of table consolidation and transformation tasks from examples. In Proc. Conference on Programming Language Design and Implementation . ACM, 422–436. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Yu Feng, Ruben Martins, Yuepeng Wang, Isil Dillig, and Thomas Reps. 2017. Component-Based Synthesis for Complex APIs. In Proc. Symposium on Principles of Programming Languages . ACM, 599–612. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. John K. Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing Data Structure Transformations from Input-output Examples. In Proc. Conference on Programming Language Design and Implementation . ACM, 229–239. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Harald Ganzinger, George Hagen, Robert Nieuwenhuis, Albert Oliveras, and Cesare Tinelli. 2004. DPLL( T): Fast Decision Procedures. In Proc. International Conference on Computer Aided Verification . Springer, 175–188.Google ScholarGoogle ScholarCross RefCross Ref
  12. Adrià Gascón, Ashish Tiwari, Brent Carmer, and Umang Mathur. 2017. Look for the Proof to Find the Program: Decorated-Component-Based Program Synthesis. In Proc. International Conference on Computer Aided Verification . Springer, 86–103.Google ScholarGoogle ScholarCross RefCross Ref
  13. Sumit Gulwani. 2011. Automating string processing in spreadsheets using input-output examples. In Proc. Symposium on Principles of Programming Languages . ACM, 317–330. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Sumit Gulwani, Susmit Jha, Ashish Tiwari, and Ramarathnam Venkatesan. 2011. Synthesis of loop-free programs. In Proc. Conference on Programming Language Design and Implementation . ACM, 62–73. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Tihomir Gvero, Viktor Kuncak, Ivan Kuraj, and Ruzica Piskac. 2013. Complete completion using types and weights. In Proc. Conference on Programming Language Design and Implementation . ACM, 27–38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Susmit Jha, Sumit Gulwani, Sanjit A. Seshia, and Ashish Tiwari. 2010. Oracle-guided component-based program synthesis. In Proc. International Conference on Software Engineering . ACM/IEEE, 215–224. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. David Mandelin, Lin Xu, Rastislav Bodík, and Doug Kimelman. 2005. Jungloid mining: helping to navigate the API jungle. In Proc. Conference on Programming Language Design and Implementation . ACM, 48–61. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Joao Marques-Silva and Karem A. Sakallah. 1999. GRASP: A Search Algorithm for Propositional Satisfiability. IEEE Trans. Computers 48, 5 (1999), 506–521. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Aditya Menon, Omer Tamuz, Sumit Gulwani, Butler Lampson, and Adam Kalai. 2013. A machine learning framework for programming by example. In Proc. International Conference on Machine Learning. Proceedings of Machine Learning Research, 187–195. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Arvind Neelakantan, Quoc V Le, Martin Abadi, Andrew McCallum, and Dario Amodei. 2017. Learning a natural language interface with neural programmer. In Proc. International Conference on Learning Representations . OpenReview.Google ScholarGoogle Scholar
  21. Arvind Neelakantan, Quoc V Le, and Ilya Sutskever. 2016. Neural programmer: Inducing latent programs with gradient descent. In Proc. International Conference on Learning Representations . OpenReview.Google ScholarGoogle Scholar
  22. Peter-Michael Osera and Steve Zdancewic. 2015. Type-and-exampledirected program synthesis. In Proc. Conference on Programming Language Design and Implementation . ACM, 619–630. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Nadia Polikarpova, Ivan Kuraj, and Armando Solar-Lezama. 2016. Program synthesis from polymorphic refinement types. Proc. Conference on Programming Language Design and Implementation (2016), 522–538. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Veselin Raychev, Pavol Bielik, Martin Vechev, and Andreas Krause. 2016. Learning programs from noisy data. In Proc. Symposium on Principles of Programming Languages . ACM, 761–774. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code completion with statistical language models. In Proc. Conference on Programming Language Design and Implementation . ACM, 419–428. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Rishabh Singh. 2016. Blinkfill: Semi-supervised programming by example for syntactic string transformations. Proceedings of the VLDB Endowment 9, 10 (2016), 816–827. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Rishabh Singh and Sumit Gulwani. 2016. Transforming spreadsheet data types using examples. In Proc. Symposium on Principles of Programming Languages . ACM, 343–356. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Calvin Smith and Aws Albarghouthi. 2016. MapReduce program synthesis. In Proc. Conference on Programming Language Design and Implementation . ACM, 326–340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Armando Solar-Lezama. 2008. Program synthesis by sketching. University of California, Berkeley.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Armando Solar-Lezama. 2009. The Sketching Approach to Program Synthesis.. In Proc. Asian Symposium on Programming Languages and Systems . Springer, 4–13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Armando Solar-Lezama, Rodric M. Rabbah, Rastislav Bodík, and Kemal Ebcioglu. 2005. Programming by sketching for bit-streaming programs. In Proc. Conference on Programming Language Design and Implementation . ACM, 281–294. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat. 2006. Combinatorial Sketching for Finite Programs. In Proc. International Conference on Architectural Support for Programming Languages and Operating Systems . ACM, 404–415. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems . 3104–3112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Ashish Tiwari, Adria Gascón, and Bruno Dutertre. 2015. Program synthesis using dual interpretation. In Proc. International Conference on Automated Deduction . Springer, 482–497.Google ScholarGoogle ScholarCross RefCross Ref
  35. Abhishek Udupa, Arun Raghavan, Jyotirmoy V Deshmukh, Sela MadorHaim, Milo MK Martin, and Rajeev Alur. 2013. TRANSIT: specifying protocols with concolic snippets. Proc. Conference on Programming Language Design and Implementation (2013), 287–296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Chenglong Wang, Alvin Cheung, and Rastislav Bodik. 2017. Synthesizing highly expressive SQL queries from input-output examples. In Proc. Conference on Programming Language Design and Implementation . ACM, 452–466. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Xinyu Wang, Isil Dillig, and Rishabh Singh. 2017. Synthesis of Data Completion Scripts using Finite Tree Automata. In Proc. International Conference on Object-Oriented Programming, Systems, Languages, and Applications . ACM, 62:1–62:26.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Xinyu Wang, Isil Dillig, and Rishabh Singh. 2018. Program Synthesis using Abstraction Refinement. In Proc. Symposium on Principles of Programming Languages . ACM, 63:1–63:30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Navid Yaghmazadeh, Christian Klinger, Isil Dillig, and Swarat Chaudhuri. 2016. Synthesizing transformations on hierarchically structured data. In Proc. Conference on Programming Language Design and Implementation . ACM, 508–521. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Navid Yaghmazadeh, Yuepeng Wang, Isil Dillig, and Thomas Dillig. 2017. SQLizer: Query Synthesis from Natural Language. In Proc. International Conference on Object-Oriented Programming, Systems, Languages, and Applications . ACM, 63:1–63:26.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Lintao Zhang, Conor F. Madigan, Matthew W. Moskewicz, and Sharad Malik. 2001. Efficient Conflict Driven Learning in Boolean Satisfiability Solver. In Proc. of International Conference on Computer-Aided Design. IEEE Computer Society, 279–285. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Sai Zhang and Yuyin Sun. 2013. Automatically synthesizing sql queries from input-output examples. In Proc. International Conference on Automated Software Engineering . IEEE, 224–234. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Program synthesis using conflict-driven learning

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 53, Issue 4
          PLDI '18
          April 2018
          834 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/3296979
          Issue’s Table of Contents
          • cover image ACM Conferences
            PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation
            June 2018
            825 pages
            ISBN:9781450356985
            DOI:10.1145/3192366

          Copyright © 2018 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 11 June 2018

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!