Abstract
This paper considers the correctness of domain-specific compilers for tensor programming languages through the study of Halide, a popular representative. It describes a translation validation algorithm for affine Halide specifications, independently of the scheduling language. The algorithm relies on “prophetic” annotations added by the compiler to the generated array assignments. The annotations provide a refinement mapping from assignments in the generated code to the tensor definitions from the specification. Our implementation leverages an affine solver and a general SMT solver, and scales to complete Halide benchmarks.
- Martin Abadi and Leslie Lamport. 1988. The Existence of Refinement Mappings. In Proceedings of the 3rd Annual Symposium on Logic in Computer Science. 165–175. https://www.microsoft.com/en-us/research/publication/the-existence-of-refinement-mappings/Google Scholar
Cross Ref
- Lénaïc Bagnères, Oleksandr Zinenko, Stéphane Huot, and Cédric Bastoul. 2016. Opening Polyhedral Compiler’s Black Box. In Proceedings of the 2016 International Symposium on Code Generation and Optimization, CGO 2016, Barcelona, Spain, March 12-18, 2016, Björn Franke, Youfeng Wu, and Fabrice Rastello (Eds.). ACM, 128–138. https://doi.org/10.1145/2854038.2854048 Google Scholar
Digital Library
- Kunal Banerjee, Chittaranjan Mandal, and Dipankar Sarkar. 2016. Translation Validation of Loop and Arithmetic Transformations in the Presence of Recurrences. SIGPLAN Not., 51, 5 (2016), Aug., 31–40. issn:0362-1340, 1558-1160 https://doi.org/10.1145/2980930.2907954 Google Scholar
Digital Library
- Wenlei Bao, Sriram Krishnamoorthy, Louis-Noël Pouchet, Fabrice Rastello, and P. Sadayappan. 2016. PolyCheck: Dynamic Verification of Iteration Space Transformations on Affine Programs. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’16). Association for Computing Machinery, New York, NY, USA. 539–554. isbn:978-1-4503-3549-2 https://doi.org/10.1145/2837614.2837656 Google Scholar
Digital Library
- Clark Barrett, Aaron Stump, and Cesare Tinelli. 2010. The SMT-LIB Standard: Version 2.0. In Proceedings of the 8th International Workshop on Satisfiability modulo Theories (Edinburgh, UK), A. Gupta and D. Kroening (Eds.).Google Scholar
- Clark W. Barrett, Yi Fang, Benjamin Goldberg, Ying Hu, Amir Pnueli, and Lenore D. Zuck. 2005. TVOC: A Translation Validator for Optimizing Compilers. In Computer Aided Verification, 17th International Conference, CAV 2005 (Lncs, Vol. 3576). springer, 291–295.Google Scholar
- Denis Barthou, Paul Feautrier, and Xavier Redon. 2002. On the Equivalence of Two Systems of Affine Recurrence Equations. In Euro-Par 2002 Parallel Processing, Burkhard Monien, Rainer Feldmann, Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen (Eds.). 2400, Springer Berlin Heidelberg, Berlin, Heidelberg. 309–313. isbn:978-3-540-44049-9 issn:0302-9743 https://doi.org/10.1007/3-540-45706-2_40 Google Scholar
Cross Ref
- G. Baumgartner, A. Auer, D.E. Bernholdt, A. Bibireata, V. Choppella, D. Cociorva, X. Gao, R.J. Harrison, S. Hirata, S. Krishnamoorthy, S. Krishnan, C. Lam, Q. Lu, M. Nooijen, R.M. Pitzer, J. Ramanujam, P. Sadayappan, and A. Sibiryakov. 2005. Synthesis of High-Performance Parallel Programs for a Class of Ab Initio Quantum Chemistry Models. Proc. IEEE, 93, 2 (2005), Feb., 276–292. issn:0018-9219 https://doi.org/10.1109/jproc.2004.840311 Google Scholar
Cross Ref
- François Bobot, Jean-Christophe Filliâtre, Claude Marché, and Andrei Paskevich. 2011. Why3: Shepherd Your Herd of Provers. In Boogie 2011: First International Workshop on Intermediate Verification Languages. Wrocł aw, Poland. 53–64.Google Scholar
- Uday Bondhugula, Muthu Baskaran, Sriram Krishnamoorthy, J. Ramanujam, A. Rountev, and P. Sadayappan. 2008. Automatic Transformations for Communication-Minimized Parallelization and Locality Optimization in the Polyhedral Model. In International Conference on Compiler Construction (ETAPS CC).Google Scholar
Digital Library
- Uday Bondhugula, Albert Hartono, J. Ramanujam, and P. Sadayappan. 2008. A Practical Automatic Polyhedral Program Optimization System. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI).Google Scholar
- Basile Clement and Albert Cohen. 2022. End-to-End Translation Validation for the Halide Language - OOPSLA’22 Artifact. Zenodo. https://doi.org/10.5281/zenodo.6390003 Google Scholar
Digital Library
- Nathanaël Courant and Xavier Leroy. 2021. Verified Code Generation for the Polyhedral Model. Proc. ACM Program. Lang., 5, POPL (2021), Article 40, Jan., 24 pages. issn:2475-1421 https://doi.org/10.1145/3434321 Google Scholar
Digital Library
- Bruno Cuervo Parrino, Julien Narboux, Eric Violard, and Nicolas Magaud. 2012. Dealing with Arithmetic Overflows in the Polyhedral Model. In IMPACT 2012 - 2nd International Workshop on Polyhedral Compilation Techniques, Uday Bondhugula and Vincent Loechner (Eds.). Paris, France. https://hal.inria.fr/hal-00655485Google Scholar
- Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Tools and Algorithms for the Construction and Analysis of Systems, C.R. Ramakrishnan and Jakob Rehof (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 337–340. isbn:978-3-540-78800-3Google Scholar
Digital Library
- Johannes Doerfert, Tobias Grosser, and Sebastian Hack. 2017. Optimistic Loop Optimization. In Proceedings of the 2017 International Symposium on Code Generation and Optimization, CGO 2017, Austin, TX, USA, February 4-8, 2017, Vijay Janapa Reddi, Aaron Smith, and Lingjia Tang (Eds.). ACM, 292–304. http://dl.acm.org/citation.cfm?id=3049864Google Scholar
Cross Ref
- P. Feautrier. 1988. Parametric Integer Programming. RAIRO Recherche Opérationnelle, 22, 3 (1988), 243–268.Google Scholar
Cross Ref
- Paul Feautrier. 1991. Dataflow Analysis of Array and Scalar References. International Journal of Parallel Programming, 20, 1 (1991), Feb., 23–53. issn:0885-7458, 1573-7640 https://doi.org/10.1007/bf01407931 Google Scholar
Cross Ref
- Tobias Grosser, Sven Verdoolaege, and Albert Cohen. 2015. Polyhedral AST Generation Is More than Scanning Polyhedra. ACM Transactions on Programming Languages and Systems, 37, 4 (2015), Article 12, July, issn:0164-0925 https://doi.org/10.1145/2743016 Google Scholar
Digital Library
- Guillaume Iooss, Christophe Alias, and Sanjay Rajopadhye. 2014. On Program Equivalence with Reductions. In Static Analysis, Markus Müller-Olm and Helmut Seidl (Eds.). 8723, Springer International Publishing, Cham. 168–183. isbn:978-3-319-10935-0 issn:0302-9743, 1611-3349 https://doi.org/10.1007/978-3-319-10936-7_11 Google Scholar
Cross Ref
- Matthieu Journault and Antoine Miné. 2018. Inferring Functional Properties of Matrix Manipulating Programs by Abstract Interpretation. Formal methods in system design, 53, 2 (2018), Feb., 221–258. issn:0925-9856, 1572-8102 https://doi.org/10.1007/s10703-017-0311-x Google Scholar
Digital Library
- Chandan Karfa, K. Banerjee, D. Sarkar, and C. Mandal. 2013. Experimentation with SMT Solvers and Theorem Provers for Verification of Loop and Arithmetic Transformations. In Proceedings of the 5th IBM Collaborative Academia Research Exchange Workshop on - i-Care ’13. ACM Press. isbn:978-1-4503-2320-8 https://doi.org/10.1145/2528228.2528231 Google Scholar
Digital Library
- Chandan Karfa, Kunal Banerjee, Dipankar Sarkar, and Chittaranjan Mandal. 2013. Verification of Loop and Arithmetic Transformations of Array-Intensive Behaviors. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst., 32, 11 (2013), Nov., 1787–1800. issn:0278-0070, 1937-4151 https://doi.org/10.1109/tcad.2013.2272536 Google Scholar
Digital Library
- Richard M. Karp, Raymond E. Miller, and Shmuel Winograd. 1967. The Organization of Computations for Uniform Recurrence Equations. Journal of The Acm, 14, 3 (1967), July, 563–590. issn:0004-5411 https://doi.org/10.1145/321406.321418 Google Scholar
Digital Library
- Sudipta Kundu, Zachary Tatlock, and Sorin Lerner. 2009. Proving Optimizations Correct Using Parameterized Program Equivalence. ACM SIGPLAN Notices, 44, 6 (2009), May, 327–337. https://doi.org/10.1145/1543135.1542513 Google Scholar
Digital Library
- Hervé Le Verge, Christophe Mauras, and Patrice Quinton. 1991. The ALPHA Language and Its Use for the Design of Systolic Arrays. Journal of VLSI signal processing systems for signal, image and video technology, 3 (1991), 173–182. https://doi.org/10.1007/BF00925828 Google Scholar
Cross Ref
- K. Rustan M. Leino. 2010. Dafny: An Automatic Program Verifier for Functional Correctness. In Proceedings of the 16th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning (LPAR’10). Springer-Verlag, Berlin, Heidelberg. 348–370. isbn:978-3-642-17510-7Google Scholar
Digital Library
- David Menendez, Santosh Nagarakatte, and Aarti Gupta. 2016. Alive-FP: Automated Verification of Floating Point Based Peephole Optimizations in LLVM. In Static Analysis, Xavier Rival (Ed.) (Lecture Notes in Computer Science). Springer, Berlin, Heidelberg. 317–337. isbn:978-3-662-53413-7 https://doi.org/10.1007/978-3-662-53413-7_16 Google Scholar
Cross Ref
- Thierry Moreau, Tianqi Chen, Luis Vega, Jared Roesch, Eddie Yan, Lianmin Zheng, Josh Fromm, Ziheng Jiang, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2019. A Hardware– Software Blueprint for Flexible Deep Learning Specialization. IEEE Micro, 39, 5 (2019), Sept., 8–16. issn:0272-1732, 1937-4143 https://doi.org/10.1109/mm.2019.2928962 Google Scholar
Cross Ref
- Ravi Teja Mullapudi, Andrew Adams, Dillon Sharlet, Jonathan Ragan-Kelley, and Kayvon Fatahalian. 2016. Automatically Scheduling Halide Image Processing Pipelines. ACM Trans. Graph., 35, 4 (2016), Article 83, July, issn:0730-0301 https://doi.org/10.1145/2897824.2925952 Google Scholar
Digital Library
- Ravi Teja Mullapudi, Vinay Vasista, and Uday Bondhugula. 2015. PolyMage: Automatic Optimization for Image Processing Pipelines. SIGPLAN Not., 50, 4 (2015), March, 429–443. issn:0362-1340 https://doi.org/10.1145/2775054.2694364 Google Scholar
Digital Library
- George C. Necula. 2000. Translation Validation for an Optimizing Compiler. In Proceedings of the ACM SIGPLAN 2000 Conference on Programming Language Design and Implementation - PLDI ’00. ACM Press, 83–95. isbn:1-58113-199-2 https://doi.org/10.1145/349299.349314 Google Scholar
Digital Library
- Amir Pnueli, Michael Siegel, and Eli Singerman. 1998. Translation Validation. In Tools and Algorithms for Construction and Analysis of Systems, TACAS ’98 (Lncs, Vol. 1384). springer, 151–166.Google Scholar
- M. Presburger. 1929. Über Die Vollständigkeit Eines Gewissen Systems Der Arithmetik Ganzer Zahlen, in Welchem Die Addition Als Einzige Operation Hervortritt. In Comptes Rendus Du Premier Congrès de Mathématiciens Des Pays Slaves. Warsaw, Poland. 92–101.Google Scholar
- Jonathan Ragan-Kelley, Andrew Adams, Sylvain Paris, Marc Levoy, Saman Amarasinghe, and Frédo Durand. 2012. Decoupling Algorithms from Schedules for Easy Optimization of Image Processing Pipelines. ACM Trans. Graph., 31, 4 (2012), Aug., 1–12. issn:0730-0301, 1557-7368 https://doi.org/10.1145/2185520.2185528 Google Scholar
Digital Library
- Jonathan Ragan-Kelley, Andrew Adams, Dillon Sharlet, Connelly Barnes, Sylvain Paris, Marc Levoy, Saman Amarasinghe, and Frédo Durand. 2017. Halide: Decoupling Algorithms from Schedules for High-Performance Image Processing. Communications of The Acm, 61, 1 (2017), Dec., 106–115. issn:0001-0782 https://doi.org/10.1145/3150211 Google Scholar
Digital Library
- Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, and Saman Amarasinghe. 2013. Halide: A Language and Compiler for Optimizing Parallelism, Locality, and Recomputation in Image Processing Pipelines. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’13). Association for Computing Machinery, New York, NY, USA. 519–530. isbn:978-1-4503-2014-6 https://doi.org/10.1145/2491956.2462176 Google Scholar
Digital Library
- Norman A. Rink and Jeronimo Castrillon. 2019. TeIL: A Type-Safe Imperative Tensor Intermediate Language. In Proceedings of the 6th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming (ARRAY 2019). Association for Computing Machinery, New York, NY, USA. 57–68. isbn:978-1-4503-6717-2 https://doi.org/10.1145/3315454.3329959 Google Scholar
Digital Library
- H. Samsom, F. Franssen, F. Catthoor, and H. De Man. 1995. System Level Verification of Video and Image Processing Specifications. In Proceedings of the 8th International Symposium on System Synthesis - ISSS 95. ACM Press. https://doi.org/10.1145/224486.224533 Google Scholar
Digital Library
- K.C. Shashidhar, Maurice Bruynooghe, Francky Catthoor, and Gerda Janssens. 2005. Verification of Source Code Transformations by Program Equivalence Checking. In Compiler Construction, Rastislav Bodik (Ed.) (Lecture Notes in Computer Science). Springer, Berlin, Heidelberg. 221–236. isbn:978-3-540-31985-6 https://doi.org/10.1007/978-3-540-31985-6_15 Google Scholar
Digital Library
- Patricia Suriana, Andrew Adams, and Shoaib Kamil. 2017. Parallel Associative Reductions in Halide. In Proceedings of the 2017 International Symposium on Code Generation and Optimization (CGO ’17). IEEE Press, Austin, USA. 281–291. isbn:978-1-5090-4931-8Google Scholar
Cross Ref
- Ross Tate, Michael Stepp, Zachary Tatlock, and Sorin Lerner. 2011. Equality Saturation: A New Approach to Optimization. Logical Methods in Computer Science, 7, 1 (2011), March, https://doi.org/10.2168/lmcs-7(1:10)2011 Google Scholar
Cross Ref
- The Coq Development Team. 2021. The Coq Proof Assistant. Zenodo. https://doi.org/10.5281/zenodo.4501022 Google Scholar
Cross Ref
- Jean-Baptiste Tristan, Paul Govereau, and Greg Morrisett. 2011. Evaluating Value-Graph Translation Validation for LLVM. ACM SIGPLAN Notices, 46, 6 (2011), June, 295–305. https://doi.org/10.1145/1993316.1993533 Google Scholar
Digital Library
- Jean-Baptiste Tristan and Xavier Leroy. 2010. A Simple, Verified Validator for Software Pipelining. ACM SIGPLAN Notices, 45, 1 (2010), Jan., 83–92. https://doi.org/10.1145/1707801.1706311 Google Scholar
Digital Library
- Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary DeVito, William S. Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. 2018. Tensor Comprehensions: Framework-agnostic High-Performance Machine Learning Abstractions. arxiv:1802.04730.Google Scholar
- Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary Devito, William S. Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. 2020. The next 700 Accelerated Layers. From Mathematical Expressions of Network Computation Graphs to Accelerated GPU Kernels, Automatically. ACM Transactions on Architecture and Code Optimization, 16, 4 (2020), Jan., 1–26. issn:1544-3566, 1544-3973 https://doi.org/10.1145/3355606 arxiv:1802.04730. Google Scholar
Digital Library
- Sven Verdoolaege. 2010. Isl: An Integer Set Library for the Polyhedral Model.. In ICMS, Komei Fukuda, Joris van der Hoeven, Michael Joswig, and Nobuki Takayama (Eds.) (Lecture Notes in Computer Science, Vol. 6327). Springer, 299–302. isbn:978-3-642-15581-9 http://dblp.uni-trier.de/db/conf/icms/icms2010.html##Verdoolaege10Google Scholar
- Sven Verdoolaege, Gerda Janssens, and Maurice Bruynooghe. 2012. Equivalence Checking of Static Affine Programs Using Widening to Handle Recurrences. ACM Transactions on Programming Languages and Systems, 34, 3 (2012), Article 11, Nov., issn:0164-0925 https://doi.org/10.1145/2362389.2362390 Google Scholar
Digital Library
- Sven Verdoolaege, Martin Palkovič, Maurice Bruynooghe, Gerda Janssens, and Francky Catthoor. 2010. Experience with Widening Based Equivalence Checking in Realistic Multimedia Systems. Journal of Electronic Testing-theory and Applications, 26, 2 (2010), Jan., 279–292. issn:0923-8174, 1573-0727 https://doi.org/10.1007/s10836-009-5140-4 Google Scholar
Digital Library
- Oleksandr Zinenko, Stéphane Huot, and Cédric Bastoul. 2018. Visual Program Manipulation in the Polyhedral Model. ACM Transactions on Architecture and Code Optimization, 15, 1 (2018), 16:1–16:25. https://doi.org/10.1145/3177961 Google Scholar
Digital Library
Index Terms
End-to-end translation validation for the halide language
Recommendations
Reflections on LMS: exploring front-end alternatives
SCALA 2016: Proceedings of the 2016 7th ACM SIGPLAN Symposium on ScalaMetaprogramming techniques to generate code at runtime in a general-purpose meta-language have seen a surge of interest in recent years, driven by the widening performance gap between high-level languages and emerging hardware platforms. In the context ...
A domain-specific language for building self-optimizing AST interpreters
GPCE '14Self-optimizing AST interpreters dynamically adapt to the provided input for faster execution. This adaptation includes initial tests of the input, changes to AST nodes, and insertion of guards that ensure assumptions still hold. Such specialization ...
iCoLa: A Compositional Meta-language with Support for Incremental Language Development
SLE 2022: Proceedings of the 15th ACM SIGPLAN International Conference on Software Language EngineeringProgramming languages providing high-level abstractions can increase programmers’ productivity and program safety. Language-oriented programming is a paradigm in which domain-specific languages are developed to solve problems within specific domains ...






Comments