Abstract
We present a new approach to example-guided program synthesis based on counterexample-guided abstraction refinement. Our method uses the abstract semantics of the underlying DSL to find a program P whose abstract behavior satisfies the examples. However, since program P may be spurious with respect to the concrete semantics, our approach iteratively refines the abstraction until we either find a program that satisfies the examples or prove that no such DSL program exists. Because many programs have the same input-output behavior in terms of their abstract semantics, this synthesis methodology significantly reduces the search space compared to existing techniques that use purely concrete semantics.
While synthesis using abstraction refinement (SYNGAR) could be implemented in different settings, we propose a refinement-based synthesis algorithm that uses abstract finite tree automata (AFTA). Our technique uses a coarse initial program abstraction to construct an initial AFTA, which is iteratively refined by constructing a proof of incorrectness of any spurious program. In addition to ruling out the spurious program accepted by the previous AFTA, proofs of incorrectness are also useful for ruling out many other spurious programs.
We implement these ideas in a framework called Blaze, which can be instantiated in different domains by providing a suitable DSL and its corresponding concrete and abstract semantics. We have used the Blaze framework to build synthesizers for string and matrix transformations, and we compare Blaze with existing techniques. Our results for the string domain show that Blaze compares favorably with FlashFill, a domain-specific synthesizer that is now deployed in Microsoft PowerShell. In the context of matrix manipulations, we compare Blaze against Prose, a state-of-the-art general-purpose VSA-based synthesizer, and show that Blaze results in a 90x speed-up over Prose. In both application domains, Blaze also consistently improves upon the performance of two other existing techniques by at least an order of magnitude.
Supplemental Material
- Parosh A Abdulla, Ahmed Bouajjani, Lukáš Holík, Lisa Kaati, and Tomáš Vojnar. 2008. Composed bisimulation for tree automata. In International Conference on Implementation and Application of Automata. Springer, 212–222. Google Scholar
Digital Library
- Aws Albarghouthi, Sumit Gulwani, and Zachary Kincaid. 2013. Recursive Program Synthesis. In International Conference on Computer Aided Verification (CAV). Springer, 934–950. Google Scholar
Digital Library
- Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo MK Martin, Mukund Raghothaman, Sanjit A Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. 2015. Syntax-guided synthesis. Dependable Software Systems Engineering 40 (2015), 1–25.Google Scholar
- Rajeev Alur, Dana Fisman, Rishabh Singh, and Armando Solar-Lezama. 2016. SyGuS-Comp 2016: Results and Analysis. In SYNT. 178–202.Google Scholar
- Rajeev Alur, Arjun Radhakrishna, and Abhishek Udupa. 2017. Scaling Enumerative Program Synthesis via Divide and Conquer. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). Springer, 319–336. Google Scholar
Cross Ref
- Thomas Ball, Vladimir Levin, and Sriram K Rajamani. 2011. A decade of software model checking with SLAM. Commun. ACM 54, 7 (2011), 68–76. Google Scholar
Digital Library
- Dirk Beyer, Thomas A Henzinger, Ranjit Jhala, and Rupak Majumdar. 2007. The software model checker Blast. International Journal on Software Tools for Technology Transfer 9, 5-6 (2007), 505–525.Google Scholar
Digital Library
- Alvin Cheung, Armando Solar-Lezama, and Samuel Madden. 2012. Using program synthesis for social recommendations. In Proceedings of the 21st ACM international conference on Information and knowledge management. ACM, 1732–1736. Google Scholar
Digital Library
- Patrick Cousot and Radhia Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In Proceedings of the 4th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (POPL). 238–252. Google Scholar
Digital Library
- Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. 2017. RobustFill: Neural Program Learning under Noisy I/O. arXiv preprint arXiv:1703.07469 (2017).Google Scholar
- Yu Feng, Ruben Martins, Jacob Van Geffen, Isil Dillig, and Swarat Chaudhuri. 2017. Component-based Synthesis of Table Consolidation and Transformation Tasks from Examples. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 422–436. Google Scholar
Digital Library
- John K. Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing Data Structure Transformations from Input-output Examples. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 229–239. Google Scholar
Digital Library
- Jonathan Frankle, Peter-Michael Osera, David Walker, and Steve Zdancewic. 2016. Example-directed Synthesis: A Typetheoretic Interpretation. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, 802–815.Google Scholar
Digital Library
- John Gallagher and German Puebla. 2002. Abstract interpretation over non-deterministic finite tree automata for set-based analysis of logic programs. Practical Aspects of Declarative Languages (2002), 243–261.Google Scholar
- Giorgio Gallo, Giustino Longo, Stefano Pallottino, and Sang Nguyen. 1993. Directed Hypergraphs and Applications. Discrete Appl. Math. 42, 2-3 (1993), 177–201.Google Scholar
Digital Library
- Adrià Gascón, Ashish Tiwari, Brent Carmer, and Umang Mathur. 2017. Look for the Proof to Find the Program: DecoratedComponent-Based Program Synthesis. In International Conference on Computer Aided Verification (CAV). Springer, 86–103. Google Scholar
Cross Ref
- Sumit Gulwani. 2011. Automating String Processing in Spreadsheets Using Input-output Examples. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, 317–330. Google Scholar
Digital Library
- Sumit Gulwani, Mikaël Mayer, Filip Niksic, and Ruzica Piskac. 2015. StriSynth: synthesis for live programming. In Proceedings of the 37th International Conference on Software Engineering (ICSE). IEEE, 701–704. Google Scholar
Cross Ref
- Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Kenneth L. McMillan. 2004. Abstractions from Proofs. In Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, 232–244. Google Scholar
Digital Library
- Thomas A Henzinger, Ranjit Jhala, Rupak Majumdar, and Grégoire Sutre. 2003. Software verification with BLAST. In International SPIN Workshop on Model Checking of Software. Springer, 235–239. Google Scholar
Cross Ref
- Haruo Hosoya and Benjamin C Pierce. 2003. XDuce: A statically typed XML processing language. ACM Transactions on Internet Technology (TOIT) 3, 2 (2003), 117–148. Google Scholar
Digital Library
- Bishoksan Kafle and John P Gallagher. 2015. Tree automata-based refinement with application to Horn clause verification. In International Workshop on Verification, Model Checking, and Abstract Interpretation (VMCAI). Springer, 209–226. Google Scholar
Digital Library
- Kevin Knight and Jonathan May. 2009. Applications of weighted automata in natural language processing. In Handbook of Weighted Automata. Springer, 571–596. Google Scholar
Cross Ref
- Alan Leung, John Sarracino, and Sorin Lerner. 2015. Interactive Parser Synthesis by Example. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 565–574. Google Scholar
Digital Library
- Parthasarathy Madhusudan. 2011. Synthesizing reactive programs. In LIPIcs-Leibniz International Proceedings in Informatics, Vol. 12. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google Scholar
- Wim Martens and Joachim Niehren. 2005. Minimizing tree automata for unranked trees. In International Workshop on Database Programming Languages. Springer, 232–246. Google Scholar
Digital Library
- Jonathan May and Kevin Knight. 2008. A Primer on Tree Automata Software for Natural Language Processing. (2008).Google Scholar
- Kenneth L McMillan and Andrey Rybalchenko. 2013. Solving constrained Horn clauses using interpolation. Tech. Rep. MSR-TR-2013-6 (2013).Google Scholar
- David Monniaux. 1999. Abstracting cryptographic protocols with tree automata. In International Static Analysis Symposium. Springer, 149–163. Google Scholar
Cross Ref
- Peter-Michael Osera and Steve Zdancewic. 2015. Type-and-example-directed Program Synthesis. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 619–630. Google Scholar
Digital Library
- Emilio Parisotto, Abdel-rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, and Pushmeet Kohli. 2016. Neurosymbolic program synthesis. arXiv preprint arXiv:1611.01855 (2016).Google Scholar
- Phitchaya Mangpo Phothilimthana, Aditya Thakur, Rastislav Bodik, and Dinakar Dhurjati. 2016. Scaling Up Superoptimization. In Proceedings of the 21st International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, 297–310. Google Scholar
Digital Library
- Nadia Polikarpova, Ivan Kuraj, and Armando Solar-Lezama. 2016. Program Synthesis from Polymorphic Refinement Types. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 522–538. Google Scholar
Digital Library
- Oleksandr Polozov and Sumit Gulwani. 2015. FlashMeta: A Framework for Inductive Program Synthesis. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). ACM, 107–126. Google Scholar
Digital Library
- Philipp Rümmer, Hossein Hojjat, and Viktor Kuncak. 2013. Classifying and solving horn clauses for verification. In Working Conference on Verified Software: Theories, Tools, and Experiments. Springer, 1–21.Google Scholar
- Gabriel Scherer and Didier Rémy. 2015. Which Simple Types Have a Unique Inhabitant?. In Proceedings of the 20th ACM SIGPLAN International Conference on Functional Programming (ICFP). ACM, 243–255. Google Scholar
Digital Library
- David E. Shaw, William R. Swartout, and C. Cordell Green. 1975. Inferring LISP Programs from Examples. In Proceedings of the 4th International Joint Conference on Artificial Intelligence (IJCAI). 260–267.Google Scholar
- Rishabh Singh. 2016. Blinkfill: Semi-supervised programming by example for syntactic string transformations. Proceedings of the VLDB Endowment 9, 10 (2016), 816–827. Google Scholar
Digital Library
- Rishabh Singh and Sumit Gulwani. 2016. Transforming Spreadsheet Data Types Using Examples. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, 343–356. Google Scholar
Digital Library
- Rishabh Singh and Armando Solar-Lezama. 2011. Synthesizing data structure manipulations from storyboards. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE). 289–299. Google Scholar
Digital Library
- Calvin Smith and Aws Albarghouthi. 2016. MapReduce Program Synthesis. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 326–340. Google Scholar
Digital Library
- Armando Solar-Lezama. 2008. Program synthesis by sketching. Ph.D. Dissertation.Google Scholar
Digital Library
- Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat. 2006. Combinatorial Sketching for Finite Programs. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, 404–415. Google Scholar
Digital Library
- James W Thatcher and Jesse B Wright. 1968. Generalized finite automata theory with an application to a decision problem of second-order logic. Theory of Computing Systems 2, 1 (1968), 57–81.Google Scholar
- Ashish Tiwari, Adrià Gascón, and Bruno Dutertre. 2015. Program Synthesis Using Dual Interpretation. In International Conference on Automated Deduction. Springer, 482–497. Google Scholar
Cross Ref
- Abhishek Udupa, Arun Raghavan, Jyotirmoy V. Deshmukh, Sela Mador-Haim, Milo M. K. Martin, and Rajeev Alur. 2013. TRANSIT: specifying protocols with concolic snippets. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). 287–296. Google Scholar
Digital Library
- Martin T. Vechev, Eran Yahav, and Greta Yorsh. 2010. Abstraction-guided synthesis of synchronization. In Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). 327–338. Google Scholar
Digital Library
- Xinyu Wang, Isil Dillig, and Rishabh Singh. 2017a. Program Synthesis using Abstraction Refinement. arXiv preprint arXiv:1710.07740 (2017).Google Scholar
- Xinyu Wang, Isil Dillig, and Rishabh Singh. 2017b. Synthesis of Data Completion Scripts Using Finite Tree Automata. Proc. ACM Program. Lang. 1, OOPSLA (Oct. 2017), 62:1–62:26. Google Scholar
Digital Library
- Xinyu Wang, Sumit Gulwani, and Rishabh Singh. 2016. FIDEX: Filtering Spreadsheet Data using Examples. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). ACM, 195–213. Google Scholar
Digital Library
- Navid Yaghmazadeh, Christian Klinger, Isil Dillig, and Swarat Chaudhuri. 2016. Synthesizing Transformations on Hierarchically Structured Data. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, 508–521. Google Scholar
Digital Library
- Yifei Yuan, Rajeev Alur, and Boon Thau Loo. 2014. NetEgg: Programming network policies by examples. In Proceedings of the 13th ACM Workshop on Hot Topics in Networks. ACM, 20. Google Scholar
Digital Library
Index Terms
Program synthesis using abstraction refinement
Recommendations
Algorithmic program synthesis: introduction
Program synthesis is a process of producing an executable program from a specification. Algorithmic synthesis produces the program automatically, without an intervention from an expert. While classical compilation falls under the definition of ...
Program synthesis from polymorphic refinement types
PLDI '16: Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and ImplementationWe present a method for synthesizing recursive functions that provably satisfy a given specification in the form of a polymorphic refinement type. We observe that such specifications are particularly suitable for program synthesis for two reasons. ...
Program synthesis by model finding
Program synthesis aims to automate the task of programming. In this paper, we present a clear and elegant formulation of program synthesis as an Alloy* specification by applying its model finder to search for a program that satisfies a contract in terms ...






Comments