Abstract
Metaprograms are programs that manipulate (generate, analyze and evaluate) other programs. These tasks are greatly facilitated by quasiquotation, a technique to construct and deconstruct program fragments using quoted code templates expressed in the syntax of the manipulated language. We argue that two main flavors of quasiquotes have existed so far: Lisp-style quasiquotes, which can both construct and deconstruct programs but may produce code that contains type mismatches and unbound variables; and MetaML-style quasiquotes, which rely on static typing to prevent these errors, but can only construct programs. In this paper, we show how to combine the advantages of both flavors into a unified framework: we allow the construction, deconstruction and evaluation of program fragments while ensuring that generated programs are well-typed and well-scoped, a combination unseen in previous work. We formalize our approach as λ{}, a multi-stage calculus with code pattern matching and rewriting, and prove its type safety. We also present its realization in Squid, a metaprogramming framework for Scala, leveraging Scala’s expressive type system. To demonstrate the usefulness of our approach, we introduce speculative rewrite rules, a novel code transformation technique that makes decisive use of these capabilities, and we outline how it simplifies the design of some crucial query compiler optimizations.
Supplemental Material
- H. Abelson, R. K. Dybvig, C. T. Haynes, G. J. Rozas, N. I. Adams, IV, D. P. Friedman, E. Kohlbecker, G. L. Steele, Jr., D. H. Bartley, R. Halstead, D. Oxley, G. J. Sussman, G. Brooks, C. Hanson, K. M. Pitman, and M. Wand. 1991. Revised4 Report on the Algorithmic Language Scheme. SIGPLAN Lisp Pointers IV, 3 (July 1991), 1–55.Google Scholar
- Yanif Ahmad and Christoph Koch. 2009. DBToaster: A SQL Compiler for High-Performance Delta Processing in MainMemory Databases. PVLDB 2, 2 (2009), 1566–1569. Google Scholar
Digital Library
- Nada Amin, Karl Samuel Grütter, Martin Odersky, Tiark Rompf, and Sandro Stucki. 2016. The Essence of Dependent Object Types. In WadlerFest 2016, Sam Lindley, Conor McBride, Phil Trinder, and Don Sannella (Eds.). Springer. Google Scholar
Cross Ref
- Michael Armbrust, Reynold S. Xin, Cheng Lian, Yin Huai, Davies Liu, Joseph K. Bradley, Xiangrui Meng, Tomer Kaftan, Michael J. Franklin, Ali Ghodsi, and Matei Zaharia. 2015. Spark SQL: Relational Data Processing in Spark. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, USA, 1383–1394. Google Scholar
Digital Library
- Emil Axelsson, Koen Claessen, Gergely Dévai, Zoltán Horváth, Karin Keijzer, Bo Lyckegård, Anders Persson, Mary Sheeran, Josef Svenningsson, and András Vajda. 2010. Feldspar: A domain specific language for digital signal processing algorithms. In 8th IEEE/ACM International Conference on Formal Methods and Models for Codesign (MEMOCODE). IEEE, 169–178. Google Scholar
Digital Library
- Alan Bawden et al. 1999. Quasiquotation in Lisp. In ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation (PEPM). ACM, 4–12.Google Scholar
- Jacques Carette, Oleg Kiselyov, and Chung-Chieh Shan. 2009. Finally tagless, partially evaluated: Tagless staged interpreters for simpler typed languages. Journal of Functional Programming 19, 05 (2009), 509–543. Google Scholar
Digital Library
- Donald D. Chamberlin, Morton M. Astrahan, Mike W. Blasgen, Jim Gray, W. Frank King III, Bruce G. Lindsay, Raymond A. Lorie, James W. Mehl, Thomas G. Price, Gianfranco R. Putzolu, Patricia G. Selinger, Mario Schkolnick, Donald R. Slutz, Irving L. Traiger, Bradford W. Wade, and Robert A. Yost. 1981. A History and Evaluation of System R. Commun. ACM 24, 10 (1981), 632–646. Google Scholar
Digital Library
- Arthur Charguéraud. 2012. The locally nameless representation. Journal of Automated Reasoning 49, 3 (2012), 363–408. Google Scholar
- James Cheney and Ralf Hinze. 2003. First-class phantom types. Technical Report. Cornell University.Google Scholar
- Andrew Crotty, Alex Galakatos, Kayhan Dursun, Tim Kraska, Ugur Çetintemel, and Stanley B. Zdonik. 2015. Tupleware:" Big" Data, Big Analytics, Small Clusters.. In 7th Biennial Conference on Innovative Data Systems Research (CIDR).Google Scholar
- Ryan Culpepper and Matthias Felleisen. 2004. Taming Macros. In Third International Conference on Generative Programming and Component Engineering (GPCE) 2004, Vancouver, Canada, October 24-28, 2004. Proceedings. Springer Berlin Heidelberg, Berlin, Heidelberg, 225–243. Google Scholar
Cross Ref
- Olivier Danvy and Andrzej Filinski. 1990. Abstracting Control. In Proceedings of the 1990 ACM Conference on LISP and Functional Programming (LFP ’90). ACM, New York, NY, USA, 151–160. Google Scholar
Digital Library
- Rowan Davies. 1996. A temporal-logic approach to binding-time analysis. In Logic in Computer Science, 1996. LICS’96. Proceedings., Eleventh Annual IEEE Symposium on. IEEE, 184–195. Google Scholar
Cross Ref
- Rowan Davies and Frank Pfenning. 2001. A modal analysis of staged computation. Journal of the ACM (JACM) 48, 3 (2001), 555–604. Google Scholar
Digital Library
- Steven E. Ganz, Amr Sabry, and Walid Taha. 2001. Macros as multi-stage computations: type-safe, generative, binding macros in MacroML. In ACM SIGPLAN Notices, Vol. 36. ACM, 74–85. Google Scholar
Digital Library
- Paolo G. Giarrusso. 2013. Open GADTs and Declaration-site Variance: A Problem Statement. In Proceedings of the 4th Workshop on Scala (SCALA ’13). ACM, New York, NY, USA, 5:1–5:4. Google Scholar
Digital Library
- Shan Shan Huang and Yannis Smaragdakis. 2011. Morphing: Structurally Shaping a Class by Reflecting on Others. ACM Trans. Program. Lang. Syst. 33, 2 (Feb. 2011).Google Scholar
Digital Library
- Shan Shan Huang, David Zook, and Yannis Smaragdakis. 2011. Statically Safe Program Generation with SafeGen. Sci. Comput. Program. 76, 5 (May 2011), 376–391. Google Scholar
Digital Library
- Paul Hudak. 1996. Building domain-specific embedded languages. ACM Computing Surveys (CSUR) 28, 4es (1996), 196.Google Scholar
Digital Library
- Paul Hudak. 1998. Modular Domain Specific Languages and Tools. In Proceedings of the 5th International Conference on Software Reuse (ICSR ’98). IEEE Computer Society, Washington, DC, USA, 134–. Google Scholar
Cross Ref
- Neil D Jones, Carsten K Gomard, and Peter Sestoft. 1993. Partial evaluation and automatic program generation. Prentice Hall.Google Scholar
Digital Library
- Simon L. Peyton Jones, Andrew Tolmach, and Tony Hoare. 2001. Playing by the rules: rewriting as a practical optimisation technique in GHC. In 2001 Haskell Workshop. ACM SIGPLAN.Google Scholar
- Vojin Jovanovic, Amir Shaikhha, Sandro Stucki, Vladimir Nikolaev, Christoph Koch, and Martin Odersky. 2014. Yin-Yang: Concealing the Deep Embedding of DSLs (GPCE 2014). ACM, 73–82.Google Scholar
- Yukiyoshi Kameyama, Oleg Kiselyov, and Chung-chieh Shan. 2014. Combinators for Impure Yet Hygienic Code Generation. In Proceedings of the ACM SIGPLAN 2014 Workshop on Partial Evaluation and Program Manipulation (PEPM ’14). ACM, New York, NY, USA, 3–14. Google Scholar
Digital Library
- Manos Karpathiotakis, Ioannis Alagiannis, Thomas Heinis, Miguel Branco, and Anastasia Ailamaki. 2015. Just-in-time data virtualization: Lightweight data management with ViDa. In CIDR.Google Scholar
- Ik-Soon Kim, Kwangkeun Yi, and Cristiano Calcagno. 2006. A polymorphic modal type system for lisp-like multi-staged languages. In ACM SIGPLAN Notices, Vol. 41. ACM, 257–268. Google Scholar
Digital Library
- Oleg Kiselyov. 2014. The design and implementation of BER MetaOCaml. In International Symposium on Functional and Logic Programming. Springer, 86–102. Google Scholar
Cross Ref
- Oleg Kiselyov. 2017. MetaOCaml – an OCaml dialect for multi-stage programming. https://web.archive.org/web/ 20170725111517/http://okmij.org/ftp/ML/MetaOCaml.htmlGoogle Scholar
- Oleg Kiselyov, Yukiyoshi Kameyama, and Yuto Sudo. 2016. Refined Environment Classifiers. In Asian Symposium on Programming Languages and Systems. Springer, 271–291. Google Scholar
Cross Ref
- Yannis Klonatos, Christoph Koch, Tiark Rompf, and Hassan Chafi. 2014. Building Efficient Query Engines in a High-Level Language. PVLDB 7, 10 (2014), 853–864. Google Scholar
Digital Library
- Christoph Koch. 2010. Incremental query evaluation in a ring of databases. In Proceedings of the Twenty-Ninth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS 2010, June 6-11, 2010, Indianapolis, Indiana, USA. ACM, 87–98. Google Scholar
Digital Library
- Christoph Koch. 2014. Abstraction Without Regret in Database Systems Building: a Manifesto. IEEE Data Eng. Bull. 37, 1 (2014), 70–79.Google Scholar
- Christoph Koch, Yanif Ahmad, Oliver Kennedy, Milos Nikolic, Andres Nötzli, Daniel Lupei, and Amir Shaikhha. 2014. DBToaster: higher-order delta processing for dynamic, frequently fresh views. The VLDB Journal 23, 2 (2014), 253–278. Google Scholar
Cross Ref
- Eugene Kohlbecker, Daniel P. Friedman, Matthias Felleisen, and Bruce Duba. 1986. Hygienic Macro Expansion. In Proceedings of the 1986 ACM Conference on LISP and Functional Programming (LFP ’86). ACM, New York, NY, USA, 151–161. Google Scholar
Digital Library
- Konstantinos Krikellas, Stratis Viglas, and Marcelo Cintra. 2010. Generating code for holistic query evaluation. In Proc. International Conference on Data Engineering (ICDE). 613–624. Google Scholar
Cross Ref
- HyoukJoong Lee, Kevin J Brown, Arvind K Sujeeth, Hassan Chafi, Tiark Rompf, Martin Odersky, and Kunle Olukotun. 2011. Implementing domain-specific languages for heterogeneous parallel computing. IEEE Micro 31, 5 (2011), 42–53.Google Scholar
Digital Library
- Geoffrey Mainland. 2007. Why It’s Nice to Be Quoted: Quasiquoting for Haskell. In Proceedings of the ACM SIGPLAN Workshop on Haskell Workshop (Haskell ’07). ACM, New York, NY, USA, 73–82. Google Scholar
Digital Library
- Conor McBride and James McKinna. 2004. Functional pearl: i am not a number–i am a free variable. In Proceedings of the 2004 ACM SIGPLAN workshop on Haskell. ACM, 1–9. Google Scholar
Digital Library
- James McKinna and Robert Pollack. 1993. Pure type systems formalized. In Typed Lambda Calculi and Applications. Springer, 289–305. Google Scholar
Cross Ref
- Fabian Nagel, Gavin Bierman, and Stratis D. Viglas. 2014. Code Generation for Efficient Query Processing in Managed Runtimes. Proc. VLDB Endow. 7, 12 (Aug. 2014), 1095–1106. Google Scholar
Digital Library
- Shayan Najd, Sam Lindley, Josef Svenningsson, and Philip Wadler. 2016. Everything Old is New Again: Quoted Domainspecific Languages. In Proceedings of the 2016 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation (PEPM 2016). ACM, New York, NY, USA, 25–36. Google Scholar
Digital Library
- Aleksandar Nanevski. 2002. Meta-programming with Names and Necessity. In Proceedings of the Seventh ACM SIGPLAN International Conference on Functional Programming (ICFP ’02). ACM, New York, NY, USA, 206–217. Google Scholar
Digital Library
- Aleksandar Nanevski and Frank Pfenning. 2005. Staged computation with names and necessity. Journal of Functional Programming 15, 6 (2005), 893–939. Google Scholar
Digital Library
- Aleksandar Nanevski, Frank Pfenning, and Brigitte Pientka. 2008. Contextual modal type theory. ACM Transactions on Computational Logic (TOCL) 9, 3 (2008), 23.Google Scholar
Digital Library
- Thomas Neumann. 2011. Efficiently Compiling Efficient Query Plans for Modern Hardware. PVLDB 4, 9 (2011), 539–550. Google Scholar
Digital Library
- Georg Ofenbeck, Tiark Rompf, Alen Stojanov, Martin Odersky, and Markus Püschel. 2013. Spiral in Scala: Towards the Systematic Construction of Generators for Performance Libraries. In Proceedings of the 12th International Conference on Generative Programming: Concepts & Experiences (GPCE ’13). ACM, New York, NY, USA, 125–134. Google Scholar
Digital Library
- Bruno C.d.S. Oliveira, Adriaan Moors, and Martin Odersky. 2010. Type Classes As Objects and Implicits. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA ’10). ACM, New York, NY, USA, 341–360. Google Scholar
Digital Library
- Lionel Parreaux, Amir Shaikhha, and Christoph E. Koch. 2017a. Quoted Staged Rewriting: a Practical Approach to LibraryDefined Optimizations. In Proceedings of the 2017 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences (GPCE 2017). ACM. Google Scholar
Digital Library
- Lionel Parreaux, Amir Shaikhha, and Christoph E. Koch. 2017b. Squid: Type-Safe, Hygienic, and Reusable Quasiquotes. In Proceedings of the 2017 8th ACM SIGPLAN Symposium on Scala (SCALA 2017). ACM. Google Scholar
Digital Library
- Frank Pfenning and Conal Elliott. 1988. Higher-order abstract syntax. In ACM SIGPLAN Notices, Vol. 23. ACM, 199–208. Google Scholar
Digital Library
- FranÃğois Pottier. 2006. An Overview of CÎśml. Electronic Notes in Theoretical Computer Science 148, 2 (2006), 27 – 52. Proceedings of the ACM-SIGPLAN Workshop on ML (ML 2005).Google Scholar
Digital Library
- WV Quine. 1940. Mathematical Logic. (1940).Google Scholar
- Morten Rhiger. 2005. First-class open and closed code fragments. In IN PROCEEDINGS OF THE SIXTH SYMPOSIUM ON TRENDS IN FUNCTIONAL PROGRAMMING.Google Scholar
- Morten Rhiger. 2012a. Hygienic quasiquotation in scheme. In Proceedings of the 2012 Annual Workshop on Scheme and Functional Programming. ACM, 58–64. Google Scholar
Digital Library
- Morten Rhiger. 2012b. Programming Languages and Systems: 21st European Symposium on Programming, ESOP 2012, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2012, Tallinn, Estonia, March 24 - April 1, 2012. Proceedings. Springer Berlin Heidelberg, Berlin, Heidelberg, Chapter Staged Computation with Staged Lexical Scope, 559–578.Google Scholar
- Tiark Rompf. 2016. Reflections on LMS: exploring front-end alternatives. In Proceedings of the 2016 7th ACM SIGPLAN Symposium on Scala. ACM, 41–50. Google Scholar
Digital Library
- Tiark Rompf and Nada Amin. 2015. Functional pearl: a SQL to C compiler in 500 lines of code. In Proceedings of the 20th ACM SIGPLAN International Conference on Functional Programming, ICFP 2015, Vancouver, BC, Canada, September 1-3, 2015. 2–9. Google Scholar
Digital Library
- Tiark Rompf and Martin Odersky. 2010. Lightweight modular staging: a pragmatic approach to runtime code generation and compiled DSLs. In Generative Programming and Component Engineering. 127–136. Google Scholar
Digital Library
- Denys Shabalin, Eugene Burmako, and Martin Odersky. 2013. Quasiquotes for Scala. Technical Report.Google Scholar
- Amir Shaikhha, Yannis Klonatos, Lionel Parreaux, Lewis Brown, Mohammad Dashti, and Christoph Koch. 2016. How to Architect a Query Compiler. In Proceedings of the 2016 International Conference on Management of Data (SIGMOD ’16). ACM, New York, NY, USA, 1907–1922. Google Scholar
Digital Library
- Tim Sheard, Zine-el-abidine Benaissa, and Emir Pasalic. 1999. DSL Implementation Using Staging and Monads. In Proceedings of the 2nd Conference on Domain-specific Languages (DSL ’99). ACM, New York, NY, USA, 81–94. Google Scholar
Digital Library
- Tim Sheard, James Hook, and Nathan Linger. 2005. GADTs+ extensible kinds= dependent programming.Google Scholar
- Tim Sheard and Simon Peyton Jones. 2002. Template meta-programming for Haskell. In Proceedings of the 2002 ACM SIGPLAN workshop on Haskell (Haskell ’02). ACM, 1–16. Google Scholar
Digital Library
- Mark R. Shinwell, Andrew M. Pitts, and Murdoch J. Gabbay. 2003. FreshML: Programming with Binders Made Simple. In Proceedings of the Eighth ACM SIGPLAN International Conference on Functional Programming (ICFP ’03). ACM, New York, NY, USA, 263–274. Google Scholar
Digital Library
- Mike Stonebraker, Daniel J. Abadi, Adam Batkin, Xuedong Chen, Mitch Cherniack, Miguel Ferreira, Edmond Lau, Amerson Lin, Sam Madden, Elizabeth O’Neil, Pat O’Neil, Alex Rasin, Nga Tran, and Stan Zdonik. 2005. C-Store: A Column-oriented DBMS. In Proceedings of the 31st International Conference on Very Large Data Bases (VLDB ’05). VLDB Endowment, 553–564.Google Scholar
Digital Library
- Donald Syme. 2006. Leveraging .NET meta-programming components from F#: integrated queries and interoperable heterogeneous execution. In Proceedings of the 2006 Workshop on ML. ACM. Google Scholar
Digital Library
- Walid Taha. 2004. Domain-Specific Program Generation: International Seminar, Dagstuhl Castle, Germany, March 23-28, 2003. Revised Papers. Springer Berlin Heidelberg, Berlin, Heidelberg, Chapter A Gentle Introduction to Multi-stage Programming, 30–50.Google Scholar
- Walid Taha and Michael Florentin Nielsen. 2003. Environment Classifiers. SIGPLAN Not. 38, 1 (Jan. 2003), 26–37. Google Scholar
Digital Library
- Walid Taha and Tim Sheard. 1997. Multi-stage programming with explicit annotations. In ACM SIGPLAN Notices, Vol. 32. ACM, 203–217. Google Scholar
Digital Library
- Walid Taha and Tim Sheard. 2000. MetaML and multi-stage programming with explicit annotations. Theor. Comput. Sci. 248, 1-2 (2000), 211–242.Google Scholar
Digital Library
- Vlad Ureche, Aggelos Biboudis, Yannis Smaragdakis, and Martin Odersky. 2015. Automating Ad Hoc Data Representation Transformations. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2015). ACM, New York, NY, USA, 801–820. Google Scholar
Digital Library
- Stratis Viglas, Gavin M. Bierman, and Fabian Nagel. 2014. Processing Declarative Queries Through Generating Imperative Code in Managed Runtimes. IEEE Data Eng. Bull. 37, 1 (2014), 12–21.Google Scholar
- Eelco Visser. 2002. Meta-programming with concrete object syntax. In Proc. International Conference on Generative Programming and Component Engineering (GPCE). Springer, 299–315. Google Scholar
Cross Ref
- Hongwei Xi, Chiyan Chen, and Gang Chen. 2003. Guarded Recursive Datatype Constructors. In Proceedings of the 30th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’03). ACM, New York, NY, USA, 224–235. Google Scholar
Digital Library
Index Terms
Unifying analytic and statically-typed quasiquotes
Recommendations
Squid: type-safe, hygienic, and reusable quasiquotes
SCALA 2017: Proceedings of the 8th ACM SIGPLAN International Symposium on ScalaQuasiquotes have been shown to greatly simplify the task of metaprogramming. This is in part because they hide the data structures of the intermediate representation (IR), instead allowing metaprogrammers to use the concrete syntax of the language they ...
MorphScala: safe class morphing with macros
SCALA '14: Proceedings of the Fifth Annual Scala WorkshopThe goal of this paper is to design an easy type-safe metaprogramming API for Scala to capture generative metaprogramming tasks that depend on existing definitions to generate others, by writing meta-code as close as possible to regular Scala code.
...
Type-Directed Elaboration of Quasiquotations: A High-Level Syntax for Low-Level Reflection
IFL '14: Proceedings of the 26nd 2014 International Symposium on Implementation and Application of Functional LanguagesIdris's reflection features allow Idris metaprograms to manipulate a representation of Idris's core language as a datatype, but these reflected terms were designed for ease of type checking and are therefore exceedingly verbose and tedious to work with. ...






Comments