skip to main content
article

Quoted staged rewriting: a practical approach to library-defined optimizations

Published:23 October 2017Publication History
Skip Abstract Section

Abstract

Staging has proved a successful technique for programmatically removing code abstractions, thereby allowing for faster program execution while retaining a high-level interface for the programmer. Unfortunately, techniques based on staging suffer from a number of problems — ranging from practicalities to fundamental limitations — which have prevented their widespread adoption. We introduce Quoted Staged Rewriting (QSR), an approach that uses type-safe, pattern matching-enabled quasiquotes to define optimizations. The approach is “staged” in two ways: first, rewrite rules can execute arbitrary code during pattern matching and code reconstruction, leveraging the power and flexibility of staging; second, library designers can orchestrate the application of successive rewriting phases (stages). The advantages of using quasiquote-based rewriting are that library designers never have to deal directly with the intermediate representation (IR), and that it allows for non-intrusive optimizations — in contrast with staging, it is not necessary to adapt the entire library and user programs to accommodate optimizations.

We show how Squid, a Scala macro-based framework, enables QSR and renders library-defined optimizations more practical than ever before: library designers write domain-specific optimizers that users invoke transparently on delimited portions of their code base. As a motivating example we describe an implementation of stream fusion (a well-known deforestation technique) that is both simpler and more powerful than the state of the art, and can readily be used by Scala programmers with no knowledge of metaprogramming.

References

  1. Emil Axelsson, Koen Claessen, Gergely Dévai, Zoltán Horváth, Karin Keijzer, Bo Lyckegård, Anders Persson, Mary Sheeran, Josef Svenningsson, and András Vajda. 2010. Feldspar: A domain specific language for digital signal processing algorithms. In Formal Methods and Models for Codesign (MEMOCODE), 2010 8th IEEE/ACM International Conference on. IEEE, 169-178. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Jacques Carette, Oleg Kiselyov, and Chung-Chieh Shan. 2009. Finally tagless, partially evaluated: Tagless staged interpreters for simpler typed languages. Journal of Functional Programming 19, 05 (2009), 509-543. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. James Cheney, Sam Lindley, and Philip Wadler. 2013. A Practical Theory of Language-integrated Query. In Proceedings of the 18th ACM SIGPLAN International Conference on Functional Programming (ICFP '13). ACM, New York, NY, USA, 403-416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Cliff Click. 1995. Global Code Motion/Global Value Numbering. In Proceedings of the ACM SIGPLAN 1995 Conference on Programming Language Design and Implementation (PLDI '95). ACM, New York, NY, USA, 246-257. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Cliff Click and Keith D. Cooper. 1995. Combining Analyses, Combining Optimizations. TOPLAS 17, 2 (March 1995), 181-196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Duncan Coutts. 2011. Stream fusion : practical shortcut fusion for coinductive sequence types. Ph.D. Dissertation. University of Oxford, UK.Google ScholarGoogle Scholar
  7. Duncan Coutts, Roman Leshchinskiy, and Don Stewart. 2007. Stream Fusion. From Lists to Streams to Nothing at All. In ICFP '07. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Oege de Moor and Ganesh Sittampalam. 1999. Generic Program Transformation. Springer Berlin Heidelberg, Berlin, Heidelberg, 116-149.Google ScholarGoogle Scholar
  9. Oege de Moor and Ganesh Sittampalam. 2001. Higher-order matching for program transformation. Theoretical Computer Science 269, 1-2 (2001), 135-162.Google ScholarGoogle ScholarCross RefCross Ref
  10. Oege de Moor and Ganesh Sittampalam. 2001. Higher-order matching for program transformation. Theoretical Computer Science 269, 1-2 (2001), 135-162.Google ScholarGoogle ScholarCross RefCross Ref
  11. Zachary DeVito, James Hegarty, Alex Aiken, Pat Hanrahan, and Jan Vitek. 2013. Terra: a multi-stage language for high-performance computing. In ACM SIGPLAN Notices, Vol. 48. ACM, 105-116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Andrew Farmer. 2015. HERMIT: Mechanized Reasoning during Compilation in the Glasgow Haskell Compiler. Ph.D. Dissertation. University of Kansas.Google ScholarGoogle Scholar
  13. Andrew Farmer, Christian Hoener zu Siederdissen, and Andy Gill. 2014. The HERMIT in the stream: fusing stream fusion's concatMap. In Proceedings of the ACM SIGPLAN 2014 workshop on Partial evaluation and program manipulation. ACM, 97-108. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Cormac Flanagan, Amr Sabry, Bruce F Duba, and Matthias Felleisen. 1993. The essence of compiling with continuations. In ACM Sigplan Notices, Vol. 28. ACM, 237-247. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Steven E Ganz, Amr Sabry, and Walid Taha. 2001. Macros as multistage computations: type-safe, generative, binding macros in MacroML. In ACM SIGPLAN Notices, Vol. 36. ACM, 74-85. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Andrew Gill, John Launchbury, and Simon L Peyton Jones. 1993. A short cut to deforestation. In Proceedings of the conference on Functional programming languages and computer architecture. ACM, 223-232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Andrew John Gill. 1996. Cheap deforestation for non-strict functional languages. Ph.D. Dissertation. University of Glasgow.Google ScholarGoogle Scholar
  18. Christian Hofer, Klaus Ostermann, Tillmann Rendel, and Adriaan Moors. 2008. Polymorphic embedding of DSLs. In Proceedings of the 7th international conference on Generative programming and component engineering. ACM, 137-148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Paul Hudak. 1996. Building domain-specific embedded languages. ACM Computing Surveys (CSUR) 28, 4es (1996), 196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Dean F Jerding, John T Stasko, and Thomas Ball. 1997. Visualizing interactions in program executions. In Proceedings of the 19th international conference on Software engineering. ACM, 360-370. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Neil D Jones, Carsten K Gomard, and Peter Sestoft. 1993. Partial evaluation and automatic program generation. Peter Sestoft. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Simon L. Peyton Jones. 1996. Compiling Haskell by program transformation: A report from the trenches. Springer Berlin Heidelberg, Berlin, Heidelberg, 18-44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Simon L. Peyton Jones, Andrew Tolmach, and Tony Hoare. [n. d.]. Playing by the rules: rewriting as a practical optimisation technique in GHC. In 2001 Haskell Workshop. ACM SIGPLAN.Google ScholarGoogle Scholar
  24. Manohar Jonnalagedda and Sandro Stucki. 2015. Fold-based Fusion As a Library: A Generative Programming Pearl. In Proceedings of the 6th ACM SIGPLAN Symposium on Scala (SCALA 2015). ACM, 41-50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Vojin Jovanovic, Amir Shaikhha, Sandro Stucki, Vladimir Nikolaev, Christoph Koch, and Martin Odersky. 2014. Yin-Yang: Concealing the Deep Embedding of DSLs (GPCE 2014). ACM, 73-82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Oleg Kiselyov, Aggelos Biboudis, Nick Palladinos, and Yannis Smaragdakis. 2017. Stream fusion, to completeness. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages. ACM, 285-299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. P. Klint, T. v. d. Storm, and J. Vinju. 2009. RASCAL: A Domain Specific Language for Source Code Analysis and Manipulation. In 2009 Ninth IEEE International Working Conference on Source Code Analysis and Manipulation. 168-177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. HyoukJoong Lee, Kevin J Brown, Arvind K Sujeeth, Hassan Chafi, Tiark Rompf, Martin Odersky, and Kunle Olukotun. 2011. Implementing domain-specific languages for heterogeneous parallel computing. IEEE Micro 31, 5 (2011), 42-53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Roland Leißa, Klaas Boesche, Sebastian Hack, Richard Membarth, and Philipp Slusallek. 2015. Shallow Embedding of DSLs via Online Partial Evaluation. In Proceedings of the 2015 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences (GPCE 2015). ACM, New York, NY, USA, 11-20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Adriaan Moors, Tiark Rompf, Philipp Haller, and Martin Odersky. 2012. Scala-virtualized. In Proceedings of the ACM SIGPLAN 2012 workshop on Partial evaluation and program manipulation. ACM, 117-120. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Shayan Najd, Sam Lindley, Josef Svenningsson, and Philip Wadler. 2016. Everything Old is New Again: Quoted Domain-specific Languages. In Proceedings of the 2016 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation (PEPM 2016). ACM, New York, NY, USA, 25-36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Georg Ofenbeck, Tiark Rompf, Alen Stojanov, Martin Odersky, and Markus Püschel. 2013. Spiral in Scala: Towards the Systematic Construction of Generators for Performance Libraries. In Proceedings of the 12th International Conference on Generative Programming: Concepts & Experiences (GPCE '13). ACM, New York, NY, USA, 125-134. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Lionel Parreaux, Amir Shaikhha, and Christoph E. Koch. 2017. Squid: Type-Safe, Hygienic, and Reusable Quasiquotes. In Proceedings of the 2017 8th ACM SIGPLAN Symposium on Scala (SCALA 2017). ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Frank Pfenning and Conal Elliott. 1988. Higher-order abstract syntax. In ACM SIGPLAN Notices, Vol. 23. ACM, 199-208. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Markus Puschel, José MF Moura, Jeremy R Johnson, David Padua, Manuela M Veloso, Bryan W Singer, Jianxin Xiong, Franz Franchetti, Aca Gacic, Yevgen Voronenko, et al. 2005. SPIRAL: Code generation for DSP transforms. Proc. IEEE 93, 2 (2005), 232-275.Google ScholarGoogle ScholarCross RefCross Ref
  36. Tiark Rompf. 2016. Reflections on LMS: exploring front-end alternatives. In Proceedings of the 2016 7th ACM SIGPLAN Symposium on Scala. ACM, 41-50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Tiark Rompf and Martin Odersky. 2010. Lightweight modular staging: a pragmatic approach to runtime code generation and compiled DSLs. In Generative Programming and Component Engineering. 127-136. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Tiark Rompf, Arvind K. Sujeeth, Nada Amin, Kevin J. Brown, Vojin Jovanovic, HyoukJoong Lee, Manohar Jonnalagedda, Kunle Olukotun, and Martin Odersky. 2013. Optimizing data structures in high-level programs: new directions for extensible compilers based on staging. In POPL. 497-510. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Tiark Rompf, Arvind K Sujeeth, HyoukJoong Lee, Kevin J Brown, Hassan Chafi, Martin Odersky, and Kunle Olukotun. 2011. Buildingblocks for performance oriented DSLs. DSL (2011).Google ScholarGoogle Scholar
  40. Lukas Rytz, Martin Odersky, and Philipp Haller. 2012. Lightweight Polymorphic Effects. In ECOOP. Springer, 258-282. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Maximilian Scherr and Shigeru Chiba. 2015. Almost First-class Language Embedding: Taming Staged Embedded DSLs. In Proceedings of the 2015 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences (GPCE 2015). ACM, New York, NY, USA, 21-30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Tim Sheard, Zine-el-abidine Benaissa, and Emir Pasalic. 1999. DSL Implementation Using Staging and Monads. In Proceedings of the 2Nd Conference on Domain-specific Languages (DSL '99). ACM, New York, NY, USA, 81-94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Anthony M. Sloane. 2011. Lightweight Language Processing in Kiama. In Proceedings of the 3rd International Summer School Conference on Generative and Transformational Techniques in Software Engineering III (GTTSE'09). Springer-Verlag, Berlin, Heidelberg, 408-425. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Michel Steuwer, Christian Fensch, Sam Lindley, and Christophe Dubach. 2015. Generating performance portable code using rewrite rules: From high-level functional expressions to high-performance opencl code. ACM SIGPLAN Notices 50, 9 (2015), 205-217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Arvind K Sujeeth, Austin Gibbons, Kevin J Brown, HyoukJoong Lee, Tiark Rompf, Martin Odersky, and Kunle Olukotun. 2013. Forge: Generating a high performance DSL implementation from a declarative specification. In Proceedings of the 12th international conference on Generative programming. ACM, 145-154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Walid Taha. 1999. Multi-stage programming: Its theory and applications. Ph.D. Dissertation. Oregon Graduate Institute of Science and Technology. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Walid Taha. 2004. Domain-Specific Program Generation: International Seminar, Dagstuhl Castle, Germany, March 23-28, 2003. Revised Papers. Springer Berlin Heidelberg, Berlin, Heidelberg, Chapter A Gentle Introduction to Multi-stage Programming, 30-50.Google ScholarGoogle Scholar
  48. Walid Taha and Tim Sheard. 1997. Multi-stage programming with explicit annotations. In ACM SIGPLAN Notices, Vol. 32. ACM, 203-217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Walid Taha and Tim Sheard. 2000. MetaML and multi-stage programming with explicit annotations. Theor. Comput. Sci. 248, 1-2 (2000), 211-242. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Vlad Ureche, Aggelos Biboudis, Yannis Smaragdakis, and Martin Odersky. 2015. Automating Ad Hoc Data Representation Transformations. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2015). ACM, New York, NY, USA, 801-820. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Eelco Visser. 2001. A survey of rewriting strategies in program transformation systems. Electronic Notes in Theoretical Computer Science 57 (2001), 109-143.Google ScholarGoogle ScholarCross RefCross Ref
  52. Eelco Visser. 2002. Meta-programming with concrete object syntax. In Generative programming and component engineering. Springer, 299-315. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Eelco Visser, Zine-el-Abidine Benaissa, and Andrew Tolmach. 1998. Building program optimizers with rewriting strategies. In ACM Sigplan Notices, Vol. 34. ACM, 13-26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Eelco Visser, Zine-el-Abidine Benaissa, and Andrew Tolmach. 1998. Building Program Optimizers with Rewriting Strategies (ICFP '98). 13-26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Philip Wadler. 1988. Deforestation: Transforming programs to eliminate trees. In ESOP'88. Springer, 344-358. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Philip Wadler. 2015. Propositions as types. Commun. ACM 58, 12 (2015), 75-84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Thomas Würthinger. 2011. Extending the graal compiler to optimize libraries. In Companion to the 26th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications. ACM, 41-42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Jeremy Yallop and Leo White. 2015. Modular macros. In OCaml Users and Developers Workshop.Google ScholarGoogle Scholar

Index Terms

  1. Quoted staged rewriting: a practical approach to library-defined optimizations

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 52, Issue 12
        GPCE '17
        December 2017
        258 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/3170492
        Issue’s Table of Contents
        • cover image ACM Conferences
          GPCE 2017: Proceedings of the 16th ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences
          October 2017
          258 pages
          ISBN:9781450355247
          DOI:10.1145/3136040

        Copyright © 2017 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 23 October 2017

        Check for updates

        Qualifiers

        • article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!