skip to main content
research-article
Open Access

Program synthesis using abstraction refinement

Published:27 December 2017Publication History
Skip Abstract Section

Abstract

We present a new approach to example-guided program synthesis based on counterexample-guided abstraction refinement. Our method uses the abstract semantics of the underlying DSL to find a program P whose abstract behavior satisfies the examples. However, since program P may be spurious with respect to the concrete semantics, our approach iteratively refines the abstraction until we either find a program that satisfies the examples or prove that no such DSL program exists. Because many programs have the same input-output behavior in terms of their abstract semantics, this synthesis methodology significantly reduces the search space compared to existing techniques that use purely concrete semantics.

While synthesis using abstraction refinement (SYNGAR) could be implemented in different settings, we propose a refinement-based synthesis algorithm that uses abstract finite tree automata (AFTA). Our technique uses a coarse initial program abstraction to construct an initial AFTA, which is iteratively refined by constructing a proof of incorrectness of any spurious program. In addition to ruling out the spurious program accepted by the previous AFTA, proofs of incorrectness are also useful for ruling out many other spurious programs.

We implement these ideas in a framework called Blaze, which can be instantiated in different domains by providing a suitable DSL and its corresponding concrete and abstract semantics. We have used the Blaze framework to build synthesizers for string and matrix transformations, and we compare Blaze with existing techniques. Our results for the string domain show that Blaze compares favorably with FlashFill, a domain-specific synthesizer that is now deployed in Microsoft PowerShell. In the context of matrix manipulations, we compare Blaze against Prose, a state-of-the-art general-purpose VSA-based synthesizer, and show that Blaze results in a 90x speed-up over Prose. In both application domains, Blaze also consistently improves upon the performance of two other existing techniques by at least an order of magnitude.

Skip Supplemental Material Section

Supplemental Material

abstractionrefinement.webm

References

  1. Parosh A Abdulla, Ahmed Bouajjani, Lukáš Holík, Lisa Kaati, and Tomáš Vojnar. 2008. Composed bisimulation for tree automata. In International Conference on Implementation and Application of Automata. Springer, 212–222. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Aws Albarghouthi, Sumit Gulwani, and Zachary Kincaid. 2013. Recursive Program Synthesis. In International Conference on Computer Aided Verification (CAV). Springer, 934–950. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo MK Martin, Mukund Raghothaman, Sanjit A Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. 2015. Syntax-guided synthesis. Dependable Software Systems Engineering 40 (2015), 1–25.Google ScholarGoogle Scholar
  4. Rajeev Alur, Dana Fisman, Rishabh Singh, and Armando Solar-Lezama. 2016. SyGuS-Comp 2016: Results and Analysis. In SYNT. 178–202.Google ScholarGoogle Scholar
  5. Rajeev Alur, Arjun Radhakrishna, and Abhishek Udupa. 2017. Scaling Enumerative Program Synthesis via Divide and Conquer. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). Springer, 319–336. Google ScholarGoogle ScholarCross RefCross Ref
  6. Thomas Ball, Vladimir Levin, and Sriram K Rajamani. 2011. A decade of software model checking with SLAM. Commun. ACM 54, 7 (2011), 68–76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Dirk Beyer, Thomas A Henzinger, Ranjit Jhala, and Rupak Majumdar. 2007. The software model checker Blast. International Journal on Software Tools for Technology Transfer 9, 5-6 (2007), 505–525.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Alvin Cheung, Armando Solar-Lezama, and Samuel Madden. 2012. Using program synthesis for social recommendations. In Proceedings of the 21st ACM international conference on Information and knowledge management. ACM, 1732–1736. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Patrick Cousot and Radhia Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In Proceedings of the 4th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (POPL). 238–252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. 2017. RobustFill: Neural Program Learning under Noisy I/O. arXiv preprint arXiv:1703.07469 (2017).Google ScholarGoogle Scholar
  11. Yu Feng, Ruben Martins, Jacob Van Geffen, Isil Dillig, and Swarat Chaudhuri. 2017. Component-based Synthesis of Table Consolidation and Transformation Tasks from Examples. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 422–436. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. John K. Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing Data Structure Transformations from Input-output Examples. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 229–239. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jonathan Frankle, Peter-Michael Osera, David Walker, and Steve Zdancewic. 2016. Example-directed Synthesis: A Typetheoretic Interpretation. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, 802–815.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. John Gallagher and German Puebla. 2002. Abstract interpretation over non-deterministic finite tree automata for set-based analysis of logic programs. Practical Aspects of Declarative Languages (2002), 243–261.Google ScholarGoogle Scholar
  15. Giorgio Gallo, Giustino Longo, Stefano Pallottino, and Sang Nguyen. 1993. Directed Hypergraphs and Applications. Discrete Appl. Math. 42, 2-3 (1993), 177–201.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Adrià Gascón, Ashish Tiwari, Brent Carmer, and Umang Mathur. 2017. Look for the Proof to Find the Program: DecoratedComponent-Based Program Synthesis. In International Conference on Computer Aided Verification (CAV). Springer, 86–103. Google ScholarGoogle ScholarCross RefCross Ref
  17. Sumit Gulwani. 2011. Automating String Processing in Spreadsheets Using Input-output Examples. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, 317–330. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Sumit Gulwani, Mikaël Mayer, Filip Niksic, and Ruzica Piskac. 2015. StriSynth: synthesis for live programming. In Proceedings of the 37th International Conference on Software Engineering (ICSE). IEEE, 701–704. Google ScholarGoogle ScholarCross RefCross Ref
  19. Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Kenneth L. McMillan. 2004. Abstractions from Proofs. In Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, 232–244. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Thomas A Henzinger, Ranjit Jhala, Rupak Majumdar, and Grégoire Sutre. 2003. Software verification with BLAST. In International SPIN Workshop on Model Checking of Software. Springer, 235–239. Google ScholarGoogle ScholarCross RefCross Ref
  21. Haruo Hosoya and Benjamin C Pierce. 2003. XDuce: A statically typed XML processing language. ACM Transactions on Internet Technology (TOIT) 3, 2 (2003), 117–148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Bishoksan Kafle and John P Gallagher. 2015. Tree automata-based refinement with application to Horn clause verification. In International Workshop on Verification, Model Checking, and Abstract Interpretation (VMCAI). Springer, 209–226. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Kevin Knight and Jonathan May. 2009. Applications of weighted automata in natural language processing. In Handbook of Weighted Automata. Springer, 571–596. Google ScholarGoogle ScholarCross RefCross Ref
  24. Alan Leung, John Sarracino, and Sorin Lerner. 2015. Interactive Parser Synthesis by Example. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 565–574. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Parthasarathy Madhusudan. 2011. Synthesizing reactive programs. In LIPIcs-Leibniz International Proceedings in Informatics, Vol. 12. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google ScholarGoogle Scholar
  26. Wim Martens and Joachim Niehren. 2005. Minimizing tree automata for unranked trees. In International Workshop on Database Programming Languages. Springer, 232–246. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Jonathan May and Kevin Knight. 2008. A Primer on Tree Automata Software for Natural Language Processing. (2008).Google ScholarGoogle Scholar
  28. Kenneth L McMillan and Andrey Rybalchenko. 2013. Solving constrained Horn clauses using interpolation. Tech. Rep. MSR-TR-2013-6 (2013).Google ScholarGoogle Scholar
  29. David Monniaux. 1999. Abstracting cryptographic protocols with tree automata. In International Static Analysis Symposium. Springer, 149–163. Google ScholarGoogle ScholarCross RefCross Ref
  30. Peter-Michael Osera and Steve Zdancewic. 2015. Type-and-example-directed Program Synthesis. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 619–630. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Emilio Parisotto, Abdel-rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, and Pushmeet Kohli. 2016. Neurosymbolic program synthesis. arXiv preprint arXiv:1611.01855 (2016).Google ScholarGoogle Scholar
  32. Phitchaya Mangpo Phothilimthana, Aditya Thakur, Rastislav Bodik, and Dinakar Dhurjati. 2016. Scaling Up Superoptimization. In Proceedings of the 21st International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, 297–310. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Nadia Polikarpova, Ivan Kuraj, and Armando Solar-Lezama. 2016. Program Synthesis from Polymorphic Refinement Types. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 522–538. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Oleksandr Polozov and Sumit Gulwani. 2015. FlashMeta: A Framework for Inductive Program Synthesis. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). ACM, 107–126. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Philipp Rümmer, Hossein Hojjat, and Viktor Kuncak. 2013. Classifying and solving horn clauses for verification. In Working Conference on Verified Software: Theories, Tools, and Experiments. Springer, 1–21.Google ScholarGoogle Scholar
  36. Gabriel Scherer and Didier Rémy. 2015. Which Simple Types Have a Unique Inhabitant?. In Proceedings of the 20th ACM SIGPLAN International Conference on Functional Programming (ICFP). ACM, 243–255. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. David E. Shaw, William R. Swartout, and C. Cordell Green. 1975. Inferring LISP Programs from Examples. In Proceedings of the 4th International Joint Conference on Artificial Intelligence (IJCAI). 260–267.Google ScholarGoogle Scholar
  38. Rishabh Singh. 2016. Blinkfill: Semi-supervised programming by example for syntactic string transformations. Proceedings of the VLDB Endowment 9, 10 (2016), 816–827. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Rishabh Singh and Sumit Gulwani. 2016. Transforming Spreadsheet Data Types Using Examples. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, 343–356. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Rishabh Singh and Armando Solar-Lezama. 2011. Synthesizing data structure manipulations from storyboards. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE). 289–299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Calvin Smith and Aws Albarghouthi. 2016. MapReduce Program Synthesis. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 326–340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Armando Solar-Lezama. 2008. Program synthesis by sketching. Ph.D. Dissertation.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat. 2006. Combinatorial Sketching for Finite Programs. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, 404–415. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. James W Thatcher and Jesse B Wright. 1968. Generalized finite automata theory with an application to a decision problem of second-order logic. Theory of Computing Systems 2, 1 (1968), 57–81.Google ScholarGoogle Scholar
  45. Ashish Tiwari, Adrià Gascón, and Bruno Dutertre. 2015. Program Synthesis Using Dual Interpretation. In International Conference on Automated Deduction. Springer, 482–497. Google ScholarGoogle ScholarCross RefCross Ref
  46. Abhishek Udupa, Arun Raghavan, Jyotirmoy V. Deshmukh, Sela Mador-Haim, Milo M. K. Martin, and Rajeev Alur. 2013. TRANSIT: specifying protocols with concolic snippets. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). 287–296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Martin T. Vechev, Eran Yahav, and Greta Yorsh. 2010. Abstraction-guided synthesis of synchronization. In Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). 327–338. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Xinyu Wang, Isil Dillig, and Rishabh Singh. 2017a. Program Synthesis using Abstraction Refinement. arXiv preprint arXiv:1710.07740 (2017).Google ScholarGoogle Scholar
  49. Xinyu Wang, Isil Dillig, and Rishabh Singh. 2017b. Synthesis of Data Completion Scripts Using Finite Tree Automata. Proc. ACM Program. Lang. 1, OOPSLA (Oct. 2017), 62:1–62:26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Xinyu Wang, Sumit Gulwani, and Rishabh Singh. 2016. FIDEX: Filtering Spreadsheet Data using Examples. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). ACM, 195–213. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Navid Yaghmazadeh, Christian Klinger, Isil Dillig, and Swarat Chaudhuri. 2016. Synthesizing Transformations on Hierarchically Structured Data. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 508–521. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Yifei Yuan, Rajeev Alur, and Boon Thau Loo. 2014. NetEgg: Programming network policies by examples. In Proceedings of the 13th ACM Workshop on Hot Topics in Networks. ACM, 20. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Program synthesis using abstraction refinement

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!