Abstract
Halide is a domain-specific language for high-performance image processing and tensor computations, widely adopted in industry. Internally, the Halide compiler relies on a term rewriting system to prove properties of code required for efficient and correct compilation. This rewrite system is a collection of handwritten transformation rules that incrementally rewrite expressions into simpler forms; the system requires high performance in both time and memory usage to keep compile times low, while operating over the undecidable theory of integers. In this work, we apply formal techniques to prove the correctness of existing rewrite rules and provide a guarantee of termination. Then, we build an automatic program synthesis system in order to craft new, provably correct rules from failure cases where the compiler was unable to prove properties. We identify and fix 4 incorrect rules as well as 8 rules which could give rise to infinite rewriting loops. We demonstrate that the synthesizer can produce better rules than hand-authored ones in five bug fixes, and describe four cases in which it has served as an assistant to a human compiler engineer. We further show that it can proactively improve weaknesses in the compiler by synthesizing a large number of rules without human supervision and showing that the enhanced ruleset lowers peak memory usage of compiled code without appreciably increasing compilation times.
Supplemental Material
- Andrew Adams, Karima Ma, Luke Anderson, Riyadh Baghdadi, Tzu-Mao Li, Michaël Gharbi, Benoit Steiner, Steven Johnson, Kayvon Fatahalian, Frédo Durand, and Jonathan Ragan-Kelley. 2019. Learning to Optimize Halide with Tree Search and Random Programs. ACM Trans. Graph. 38, 4, Article 121 ( July 2019 ), 12 pages. https://doi.org/10.1145/3306346.3322967 Google Scholar
Digital Library
- Angello Astorga, P. Madhusudan, Shambwaditya Saha, Shiyu Wang, and Tao Xie. 2019. Learning Stateful Preconditions modulo a Test Generator. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (Phoenix, AZ, USA) ( PLDI 2019 ). Association for Computing Machinery, New York, NY, USA.Google Scholar
Digital Library
- Franz Baader and Tobias Nipkow. 1999. Term rewriting and all that. Cambridge university press.Google Scholar
Digital Library
- Raymond T Boute. 1992. The Euclidean definition of the functions div and mod. ACM Transactions on Programming Languages and Systems (TOPLAS) 14, 2 ( 1992 ), 127-144.Google Scholar
Digital Library
- Eric Butler, Emina Torlak, and Zoran Popović. 2017. Synthesizing interpretable strategies for solving puzzle games. In Proceedings of the 12th International Conference on the Foundations of Digital Games. ACM, 10.Google Scholar
Digital Library
- Eric Butler, Emina Torlak, and Zoran Popović. 2018. A Framework for Computer-Aided Design of Educational Domain Models. In International Conference on Verification, Model Checking, and Abstract Interpretation. Springer, 138-160.Google Scholar
Cross Ref
- Alessandro Cimatti, Alberto Griggio, Bastiaan Schaafsma, and Roberto Sebastiani. 2013. The MathSAT5 SMT Solver. In Proceedings of TACAS (LNCS), Nir Piterman and Scott Smolka (Eds.), Vol. 7795. Springer.Google Scholar
Digital Library
- The Coq Development Team. 2019. The Coq Reference Manual, version 8.10. Available electronically at http://coq.inria.fr/doc.Google Scholar
- Patrick Cousot and Radhia Cousot. 1977. Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In Proceedings of the 4th ACM SIGACT-SIGPLAN symposium on Principles of programming languages. ACM, 238-252.Google Scholar
Digital Library
- Patrick Cousot and Radhia Cousot. 1979. Systematic design of program analysis frameworks. In Proceedings of the 6th ACM SIGACT-SIGPLAN symposium on Principles of programming languages. ACM, 269-282.Google Scholar
Digital Library
- Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An eficient SMT solver. In International conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, 337-340.Google Scholar
Cross Ref
- John K. Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing Data Structure Transformations from Input-Output Examples. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (Portland, OR, USA) ( PLDI '15). Association for Computing Machinery, New York, NY, USA, 11.Google Scholar
Digital Library
- Saul Gorn. 1967. Handling the Growth by Definition of Mechanical Languages. In Proceedings of the April 18-20, 1967, Spring Joint Computer Conference (Atlantic City, New Jersey) (AFIPS '67 (Spring)). ACM, New York, NY, USA, 213-224. https://doi.org/10.1145/1465482.1465513 Google Scholar
Digital Library
- Bastian Hagedorn, Johannes Lenfers, Thomas Koehler, Xueying Qin, Sergei Gorlatch, and Michel Steuwer. 2020. Achieving high-performance the functional way: a functional pearl on expressing high-performance optimizations as rewrite strategies. Proceedings of the ACM on Programming Languages 4, ICFP ( 2020 ), 1-29.Google Scholar
Digital Library
- Dejan Jovanović. 2017. Solving nonlinear integer arithmetic with MCSAT. In International Conference on Verification, Model Checking, and Abstract Interpretation. Springer, 330-346.Google Scholar
Cross Ref
- Donald E Knuth and Peter B Bendix. 1983. Simple word problems in universal algebras. In Automation of Reasoning. Springer, 342-376.Google Scholar
- DongKwon Lee, Woosuk Lee, Hakjoo Oh, and Kwangkeun Yi. 2020. Optimizing Homomorphic Evaluation Circuits by Program Synthesis and Term Rewriting. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2020 ).Google Scholar
Digital Library
- Nuno P. Lopes, David Menendez, Santosh Nagarakatte, and John Regehr. 2015. Provably Correct Peephole Optimizations with Alive. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (Portland, OR, USA) ( PLDI '15). Association for Computing Machinery, New York, NY, USA, 22-32.Google Scholar
Digital Library
- Nuno P Lopes and José Monteiro. 2014. Weakest precondition synthesis for compiler optimizations. In International Conference on Verification, Model Checking, and Abstract Interpretation. Springer, 203-221.Google Scholar
Digital Library
- Henry Massalin. 1987. Superoptimizer: a look at the smallest program. ACM SIGARCH Computer Architecture News 15, 5 ( 1987 ), 122-126.Google Scholar
Digital Library
- David Menendez and Santosh Nagarakatte. 2017. Alive-Infer: Data-Driven Precondition Inference for Peephole Optimizations in LLVM. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation (Barcelona, Spain) ( PLDI 2017 ). Association for Computing Machinery, New York, NY, USA, 49-63. https://doi.org/10. 1145/3062341.3062372 Google Scholar
Digital Library
- C.G. Nelson. 1980. Techniques for program verification[ Ph. D. Thesis]. ( 1980 ).Google Scholar
- Pavel Panchekha, Alex Sanchez-Stern, James R Wilcox, and Zachary Tatlock. 2015. Automatically improving accuracy for lfoating point expressions. In ACM SIGPLAN Notices, Vol. 50. ACM, 1-11.Google Scholar
- Phitchaya Mangpo Phothilimthana, Aditya Thakur, Rastislav Bodik, and Dinakar Dhurjati. 2016a. Scaling Up Superoptimization. In Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems (Atlanta, Georgia, USA) ( ASPLOS '16). ACM, New York, NY, USA, 297-310. https://doi.org/10.1145/2872362.2872387 Google Scholar
Digital Library
- Phitchaya Mangpo Phothilimthana, Aditya Thakur, Rastislav Bodik, and Dinakar Dhurjati. 2016b. Scaling up superoptimization. In ACM SIGARCH Computer Architecture News, Vol. 44. ACM, 297-310.Google Scholar
Digital Library
- Alex Reinking. 2019. Formal Semantics for the Halide Language. Master's thesis. University of California at Berkeley.Google Scholar
- Raimondas Sasnauskas, Yang Chen, Peter Collingbourne, Jeroen Ketema, Gratian Lup, Jubi Taneja, and John Regehr. 2017a. Souper: A Synthesizing Superoptimizer. arXiv: 1711.04422 [cs.PL]Google Scholar
- Raimondas Sasnauskas, Yang Chen, Peter Collingbourne, Jeroen Ketema, Jubi Taneja, and John Regehr. 2017b. Souper: A Synthesizing Superoptimizer. CoRR abs/1711.04422 ( 2017 ). arXiv: 1711.04422 http://arxiv.org/abs/1711.04422Google Scholar
- Eric Schkufza, Rahul Sharma, and Alex Aiken. 2013. Stochastic superoptimization. In ACM SIGPLAN Notices, Vol. 48. ACM, 305-316.Google Scholar
Digital Library
- Rohit Singh and Armando Solar-Lezama. 2016. SWAPPER: A framework for automatic generation of formula simplifiers based on conditional rewrite rules. In 2016 Formal Methods in Computer-Aided Design (FMCAD). IEEE, 185-192.Google Scholar
- Armando Solar-Lezama. 2009. The Sketching Approach to Program Synthesis. In Programming Languages and Systems, 7th Asian Symposium, APLAS 2009, Seoul, Korea, December 14-16, 2009. Proceedings (Lecture Notes in Computer Science), Zhenjiang Hu (Ed.), Vol. 5904. Springer, 4-13. https://doi.org/10.1007/978-3-642-10672-9_3 Google Scholar
Digital Library
- Emina Torlak and Rastislav Bodik. 2014. A lightweight symbolic virtual machine for solver-aided host languages. ACM SIGPLAN Notices 49, 6 ( 2014 ), 530-541.Google Scholar
Index Terms
Verifying and improving Halide’s term rewriting system with program synthesis
Recommendations
Template-based program verification and program synthesis
Program verification is the task of automatically generating proofs for a program's compliance with a given specification. Program synthesis is the task of automatically generating a program that meets a given specification. Both program verification ...
On Term-Graph Rewrite Strategies
We tackle the problem of cyclic term-graph rewriting. We first revisit the classical algorithmic approach to term-graph rewriting by providing a definition of rewrite rules of the form lhs->rhs where the left-hand sides are term-graphs and the right-...
Program Transformation with Scoped Dynamic Rewrite Rules
Program Transformation: Theoretical Foundations and Basic Techniques. Part 2The applicability of term rewriting to program transformation is limited by the lack of control over rule application and by the context-free nature of rewrite rules. The first problem is addressed by languages supporting user-definable rewriting ...






Comments