skip to main content
research-article
Open Access
Artifacts Evaluated & Functional

Unifying analytic and statically-typed quasiquotes

Published:27 December 2017Publication History
Skip Abstract Section

Abstract

Metaprograms are programs that manipulate (generate, analyze and evaluate) other programs. These tasks are greatly facilitated by quasiquotation, a technique to construct and deconstruct program fragments using quoted code templates expressed in the syntax of the manipulated language. We argue that two main flavors of quasiquotes have existed so far: Lisp-style quasiquotes, which can both construct and deconstruct programs but may produce code that contains type mismatches and unbound variables; and MetaML-style quasiquotes, which rely on static typing to prevent these errors, but can only construct programs. In this paper, we show how to combine the advantages of both flavors into a unified framework: we allow the construction, deconstruction and evaluation of program fragments while ensuring that generated programs are well-typed and well-scoped, a combination unseen in previous work. We formalize our approach as λ{}, a multi-stage calculus with code pattern matching and rewriting, and prove its type safety. We also present its realization in Squid, a metaprogramming framework for Scala, leveraging Scala’s expressive type system. To demonstrate the usefulness of our approach, we introduce speculative rewrite rules, a novel code transformation technique that makes decisive use of these capabilities, and we outline how it simplifies the design of some crucial query compiler optimizations.

Skip Supplemental Material Section

Supplemental Material

quasiquotes.webm

References

  1. H. Abelson, R. K. Dybvig, C. T. Haynes, G. J. Rozas, N. I. Adams, IV, D. P. Friedman, E. Kohlbecker, G. L. Steele, Jr., D. H. Bartley, R. Halstead, D. Oxley, G. J. Sussman, G. Brooks, C. Hanson, K. M. Pitman, and M. Wand. 1991. Revised4 Report on the Algorithmic Language Scheme. SIGPLAN Lisp Pointers IV, 3 (July 1991), 1–55.Google ScholarGoogle Scholar
  2. Yanif Ahmad and Christoph Koch. 2009. DBToaster: A SQL Compiler for High-Performance Delta Processing in MainMemory Databases. PVLDB 2, 2 (2009), 1566–1569. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Nada Amin, Karl Samuel Grütter, Martin Odersky, Tiark Rompf, and Sandro Stucki. 2016. The Essence of Dependent Object Types. In WadlerFest 2016, Sam Lindley, Conor McBride, Phil Trinder, and Don Sannella (Eds.). Springer. Google ScholarGoogle ScholarCross RefCross Ref
  4. Michael Armbrust, Reynold S. Xin, Cheng Lian, Yin Huai, Davies Liu, Joseph K. Bradley, Xiangrui Meng, Tomer Kaftan, Michael J. Franklin, Ali Ghodsi, and Matei Zaharia. 2015. Spark SQL: Relational Data Processing in Spark. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, USA, 1383–1394. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Emil Axelsson, Koen Claessen, Gergely Dévai, Zoltán Horváth, Karin Keijzer, Bo Lyckegård, Anders Persson, Mary Sheeran, Josef Svenningsson, and András Vajda. 2010. Feldspar: A domain specific language for digital signal processing algorithms. In 8th IEEE/ACM International Conference on Formal Methods and Models for Codesign (MEMOCODE). IEEE, 169–178. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Alan Bawden et al. 1999. Quasiquotation in Lisp. In ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation (PEPM). ACM, 4–12.Google ScholarGoogle Scholar
  7. Jacques Carette, Oleg Kiselyov, and Chung-Chieh Shan. 2009. Finally tagless, partially evaluated: Tagless staged interpreters for simpler typed languages. Journal of Functional Programming 19, 05 (2009), 509–543. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Donald D. Chamberlin, Morton M. Astrahan, Mike W. Blasgen, Jim Gray, W. Frank King III, Bruce G. Lindsay, Raymond A. Lorie, James W. Mehl, Thomas G. Price, Gianfranco R. Putzolu, Patricia G. Selinger, Mario Schkolnick, Donald R. Slutz, Irving L. Traiger, Bradford W. Wade, and Robert A. Yost. 1981. A History and Evaluation of System R. Commun. ACM 24, 10 (1981), 632–646. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Arthur Charguéraud. 2012. The locally nameless representation. Journal of Automated Reasoning 49, 3 (2012), 363–408. Google ScholarGoogle Scholar
  10. James Cheney and Ralf Hinze. 2003. First-class phantom types. Technical Report. Cornell University.Google ScholarGoogle Scholar
  11. Andrew Crotty, Alex Galakatos, Kayhan Dursun, Tim Kraska, Ugur Çetintemel, and Stanley B. Zdonik. 2015. Tupleware:" Big" Data, Big Analytics, Small Clusters.. In 7th Biennial Conference on Innovative Data Systems Research (CIDR).Google ScholarGoogle Scholar
  12. Ryan Culpepper and Matthias Felleisen. 2004. Taming Macros. In Third International Conference on Generative Programming and Component Engineering (GPCE) 2004, Vancouver, Canada, October 24-28, 2004. Proceedings. Springer Berlin Heidelberg, Berlin, Heidelberg, 225–243. Google ScholarGoogle ScholarCross RefCross Ref
  13. Olivier Danvy and Andrzej Filinski. 1990. Abstracting Control. In Proceedings of the 1990 ACM Conference on LISP and Functional Programming (LFP ’90). ACM, New York, NY, USA, 151–160. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Rowan Davies. 1996. A temporal-logic approach to binding-time analysis. In Logic in Computer Science, 1996. LICS’96. Proceedings., Eleventh Annual IEEE Symposium on. IEEE, 184–195. Google ScholarGoogle ScholarCross RefCross Ref
  15. Rowan Davies and Frank Pfenning. 2001. A modal analysis of staged computation. Journal of the ACM (JACM) 48, 3 (2001), 555–604. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Steven E. Ganz, Amr Sabry, and Walid Taha. 2001. Macros as multi-stage computations: type-safe, generative, binding macros in MacroML. In ACM SIGPLAN Notices, Vol. 36. ACM, 74–85. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Paolo G. Giarrusso. 2013. Open GADTs and Declaration-site Variance: A Problem Statement. In Proceedings of the 4th Workshop on Scala (SCALA ’13). ACM, New York, NY, USA, 5:1–5:4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Shan Shan Huang and Yannis Smaragdakis. 2011. Morphing: Structurally Shaping a Class by Reflecting on Others. ACM Trans. Program. Lang. Syst. 33, 2 (Feb. 2011).Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Shan Shan Huang, David Zook, and Yannis Smaragdakis. 2011. Statically Safe Program Generation with SafeGen. Sci. Comput. Program. 76, 5 (May 2011), 376–391. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Paul Hudak. 1996. Building domain-specific embedded languages. ACM Computing Surveys (CSUR) 28, 4es (1996), 196.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Paul Hudak. 1998. Modular Domain Specific Languages and Tools. In Proceedings of the 5th International Conference on Software Reuse (ICSR ’98). IEEE Computer Society, Washington, DC, USA, 134–. Google ScholarGoogle ScholarCross RefCross Ref
  22. Neil D Jones, Carsten K Gomard, and Peter Sestoft. 1993. Partial evaluation and automatic program generation. Prentice Hall.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Simon L. Peyton Jones, Andrew Tolmach, and Tony Hoare. 2001. Playing by the rules: rewriting as a practical optimisation technique in GHC. In 2001 Haskell Workshop. ACM SIGPLAN.Google ScholarGoogle Scholar
  24. Vojin Jovanovic, Amir Shaikhha, Sandro Stucki, Vladimir Nikolaev, Christoph Koch, and Martin Odersky. 2014. Yin-Yang: Concealing the Deep Embedding of DSLs (GPCE 2014). ACM, 73–82.Google ScholarGoogle Scholar
  25. Yukiyoshi Kameyama, Oleg Kiselyov, and Chung-chieh Shan. 2014. Combinators for Impure Yet Hygienic Code Generation. In Proceedings of the ACM SIGPLAN 2014 Workshop on Partial Evaluation and Program Manipulation (PEPM ’14). ACM, New York, NY, USA, 3–14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Manos Karpathiotakis, Ioannis Alagiannis, Thomas Heinis, Miguel Branco, and Anastasia Ailamaki. 2015. Just-in-time data virtualization: Lightweight data management with ViDa. In CIDR.Google ScholarGoogle Scholar
  27. Ik-Soon Kim, Kwangkeun Yi, and Cristiano Calcagno. 2006. A polymorphic modal type system for lisp-like multi-staged languages. In ACM SIGPLAN Notices, Vol. 41. ACM, 257–268. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Oleg Kiselyov. 2014. The design and implementation of BER MetaOCaml. In International Symposium on Functional and Logic Programming. Springer, 86–102. Google ScholarGoogle ScholarCross RefCross Ref
  29. Oleg Kiselyov. 2017. MetaOCaml – an OCaml dialect for multi-stage programming. https://web.archive.org/web/ 20170725111517/http://okmij.org/ftp/ML/MetaOCaml.htmlGoogle ScholarGoogle Scholar
  30. Oleg Kiselyov, Yukiyoshi Kameyama, and Yuto Sudo. 2016. Refined Environment Classifiers. In Asian Symposium on Programming Languages and Systems. Springer, 271–291. Google ScholarGoogle ScholarCross RefCross Ref
  31. Yannis Klonatos, Christoph Koch, Tiark Rompf, and Hassan Chafi. 2014. Building Efficient Query Engines in a High-Level Language. PVLDB 7, 10 (2014), 853–864. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Christoph Koch. 2010. Incremental query evaluation in a ring of databases. In Proceedings of the Twenty-Ninth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS 2010, June 6-11, 2010, Indianapolis, Indiana, USA. ACM, 87–98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Christoph Koch. 2014. Abstraction Without Regret in Database Systems Building: a Manifesto. IEEE Data Eng. Bull. 37, 1 (2014), 70–79.Google ScholarGoogle Scholar
  34. Christoph Koch, Yanif Ahmad, Oliver Kennedy, Milos Nikolic, Andres Nötzli, Daniel Lupei, and Amir Shaikhha. 2014. DBToaster: higher-order delta processing for dynamic, frequently fresh views. The VLDB Journal 23, 2 (2014), 253–278. Google ScholarGoogle ScholarCross RefCross Ref
  35. Eugene Kohlbecker, Daniel P. Friedman, Matthias Felleisen, and Bruce Duba. 1986. Hygienic Macro Expansion. In Proceedings of the 1986 ACM Conference on LISP and Functional Programming (LFP ’86). ACM, New York, NY, USA, 151–161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Konstantinos Krikellas, Stratis Viglas, and Marcelo Cintra. 2010. Generating code for holistic query evaluation. In Proc. International Conference on Data Engineering (ICDE). 613–624. Google ScholarGoogle ScholarCross RefCross Ref
  37. HyoukJoong Lee, Kevin J Brown, Arvind K Sujeeth, Hassan Chafi, Tiark Rompf, Martin Odersky, and Kunle Olukotun. 2011. Implementing domain-specific languages for heterogeneous parallel computing. IEEE Micro 31, 5 (2011), 42–53.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Geoffrey Mainland. 2007. Why It’s Nice to Be Quoted: Quasiquoting for Haskell. In Proceedings of the ACM SIGPLAN Workshop on Haskell Workshop (Haskell ’07). ACM, New York, NY, USA, 73–82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Conor McBride and James McKinna. 2004. Functional pearl: i am not a number–i am a free variable. In Proceedings of the 2004 ACM SIGPLAN workshop on Haskell. ACM, 1–9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. James McKinna and Robert Pollack. 1993. Pure type systems formalized. In Typed Lambda Calculi and Applications. Springer, 289–305. Google ScholarGoogle ScholarCross RefCross Ref
  41. Fabian Nagel, Gavin Bierman, and Stratis D. Viglas. 2014. Code Generation for Efficient Query Processing in Managed Runtimes. Proc. VLDB Endow. 7, 12 (Aug. 2014), 1095–1106. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Shayan Najd, Sam Lindley, Josef Svenningsson, and Philip Wadler. 2016. Everything Old is New Again: Quoted Domainspecific Languages. In Proceedings of the 2016 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation (PEPM 2016). ACM, New York, NY, USA, 25–36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Aleksandar Nanevski. 2002. Meta-programming with Names and Necessity. In Proceedings of the Seventh ACM SIGPLAN International Conference on Functional Programming (ICFP ’02). ACM, New York, NY, USA, 206–217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Aleksandar Nanevski and Frank Pfenning. 2005. Staged computation with names and necessity. Journal of Functional Programming 15, 6 (2005), 893–939. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Aleksandar Nanevski, Frank Pfenning, and Brigitte Pientka. 2008. Contextual modal type theory. ACM Transactions on Computational Logic (TOCL) 9, 3 (2008), 23.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Thomas Neumann. 2011. Efficiently Compiling Efficient Query Plans for Modern Hardware. PVLDB 4, 9 (2011), 539–550. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Georg Ofenbeck, Tiark Rompf, Alen Stojanov, Martin Odersky, and Markus Püschel. 2013. Spiral in Scala: Towards the Systematic Construction of Generators for Performance Libraries. In Proceedings of the 12th International Conference on Generative Programming: Concepts & Experiences (GPCE ’13). ACM, New York, NY, USA, 125–134. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Bruno C.d.S. Oliveira, Adriaan Moors, and Martin Odersky. 2010. Type Classes As Objects and Implicits. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA ’10). ACM, New York, NY, USA, 341–360. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Lionel Parreaux, Amir Shaikhha, and Christoph E. Koch. 2017a. Quoted Staged Rewriting: a Practical Approach to LibraryDefined Optimizations. In Proceedings of the 2017 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences (GPCE 2017). ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Lionel Parreaux, Amir Shaikhha, and Christoph E. Koch. 2017b. Squid: Type-Safe, Hygienic, and Reusable Quasiquotes. In Proceedings of the 2017 8th ACM SIGPLAN Symposium on Scala (SCALA 2017). ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Frank Pfenning and Conal Elliott. 1988. Higher-order abstract syntax. In ACM SIGPLAN Notices, Vol. 23. ACM, 199–208. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. FranÃğois Pottier. 2006. An Overview of CÎśml. Electronic Notes in Theoretical Computer Science 148, 2 (2006), 27 – 52. Proceedings of the ACM-SIGPLAN Workshop on ML (ML 2005).Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. WV Quine. 1940. Mathematical Logic. (1940).Google ScholarGoogle Scholar
  54. Morten Rhiger. 2005. First-class open and closed code fragments. In IN PROCEEDINGS OF THE SIXTH SYMPOSIUM ON TRENDS IN FUNCTIONAL PROGRAMMING.Google ScholarGoogle Scholar
  55. Morten Rhiger. 2012a. Hygienic quasiquotation in scheme. In Proceedings of the 2012 Annual Workshop on Scheme and Functional Programming. ACM, 58–64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Morten Rhiger. 2012b. Programming Languages and Systems: 21st European Symposium on Programming, ESOP 2012, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2012, Tallinn, Estonia, March 24 - April 1, 2012. Proceedings. Springer Berlin Heidelberg, Berlin, Heidelberg, Chapter Staged Computation with Staged Lexical Scope, 559–578.Google ScholarGoogle Scholar
  57. Tiark Rompf. 2016. Reflections on LMS: exploring front-end alternatives. In Proceedings of the 2016 7th ACM SIGPLAN Symposium on Scala. ACM, 41–50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Tiark Rompf and Nada Amin. 2015. Functional pearl: a SQL to C compiler in 500 lines of code. In Proceedings of the 20th ACM SIGPLAN International Conference on Functional Programming, ICFP 2015, Vancouver, BC, Canada, September 1-3, 2015. 2–9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Tiark Rompf and Martin Odersky. 2010. Lightweight modular staging: a pragmatic approach to runtime code generation and compiled DSLs. In Generative Programming and Component Engineering. 127–136. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Denys Shabalin, Eugene Burmako, and Martin Odersky. 2013. Quasiquotes for Scala. Technical Report.Google ScholarGoogle Scholar
  61. Amir Shaikhha, Yannis Klonatos, Lionel Parreaux, Lewis Brown, Mohammad Dashti, and Christoph Koch. 2016. How to Architect a Query Compiler. In Proceedings of the 2016 International Conference on Management of Data (SIGMOD ’16). ACM, New York, NY, USA, 1907–1922. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Tim Sheard, Zine-el-abidine Benaissa, and Emir Pasalic. 1999. DSL Implementation Using Staging and Monads. In Proceedings of the 2nd Conference on Domain-specific Languages (DSL ’99). ACM, New York, NY, USA, 81–94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Tim Sheard, James Hook, and Nathan Linger. 2005. GADTs+ extensible kinds= dependent programming.Google ScholarGoogle Scholar
  64. Tim Sheard and Simon Peyton Jones. 2002. Template meta-programming for Haskell. In Proceedings of the 2002 ACM SIGPLAN workshop on Haskell (Haskell ’02). ACM, 1–16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Mark R. Shinwell, Andrew M. Pitts, and Murdoch J. Gabbay. 2003. FreshML: Programming with Binders Made Simple. In Proceedings of the Eighth ACM SIGPLAN International Conference on Functional Programming (ICFP ’03). ACM, New York, NY, USA, 263–274. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Mike Stonebraker, Daniel J. Abadi, Adam Batkin, Xuedong Chen, Mitch Cherniack, Miguel Ferreira, Edmond Lau, Amerson Lin, Sam Madden, Elizabeth O’Neil, Pat O’Neil, Alex Rasin, Nga Tran, and Stan Zdonik. 2005. C-Store: A Column-oriented DBMS. In Proceedings of the 31st International Conference on Very Large Data Bases (VLDB ’05). VLDB Endowment, 553–564.Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Donald Syme. 2006. Leveraging .NET meta-programming components from F#: integrated queries and interoperable heterogeneous execution. In Proceedings of the 2006 Workshop on ML. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Walid Taha. 2004. Domain-Specific Program Generation: International Seminar, Dagstuhl Castle, Germany, March 23-28, 2003. Revised Papers. Springer Berlin Heidelberg, Berlin, Heidelberg, Chapter A Gentle Introduction to Multi-stage Programming, 30–50.Google ScholarGoogle Scholar
  69. Walid Taha and Michael Florentin Nielsen. 2003. Environment Classifiers. SIGPLAN Not. 38, 1 (Jan. 2003), 26–37. Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Walid Taha and Tim Sheard. 1997. Multi-stage programming with explicit annotations. In ACM SIGPLAN Notices, Vol. 32. ACM, 203–217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Walid Taha and Tim Sheard. 2000. MetaML and multi-stage programming with explicit annotations. Theor. Comput. Sci. 248, 1-2 (2000), 211–242.Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Vlad Ureche, Aggelos Biboudis, Yannis Smaragdakis, and Martin Odersky. 2015. Automating Ad Hoc Data Representation Transformations. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2015). ACM, New York, NY, USA, 801–820. Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Stratis Viglas, Gavin M. Bierman, and Fabian Nagel. 2014. Processing Declarative Queries Through Generating Imperative Code in Managed Runtimes. IEEE Data Eng. Bull. 37, 1 (2014), 12–21.Google ScholarGoogle Scholar
  74. Eelco Visser. 2002. Meta-programming with concrete object syntax. In Proc. International Conference on Generative Programming and Component Engineering (GPCE). Springer, 299–315. Google ScholarGoogle ScholarCross RefCross Ref
  75. Hongwei Xi, Chiyan Chen, and Gang Chen. 2003. Guarded Recursive Datatype Constructors. In Proceedings of the 30th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’03). ACM, New York, NY, USA, 224–235. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Unifying analytic and statically-typed quasiquotes

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!