skip to main content
research-article

Lightweight modular staging: a pragmatic approach to runtime code generation and compiled DSLs

Authors Info & Claims
Published:10 October 2010Publication History
Skip Abstract Section

Abstract

Software engineering demands generality and abstraction, performance demands specialization and concretization. Generative programming can provide both, but the effort required to develop high-quality program generators likely offsets their benefits, even if a multi-stage programming language is used.

We present lightweight modular staging, a library-based multi-stage programming approach that breaks with the tradition of syntactic quasi-quotation and instead uses only types to distinguish between binding times. Through extensive use of component technology, lightweight modular staging makes an optimizing compiler framework available at the library level, allowing programmers to tightly integrate domain-specific abstractions and optimizations into the generation process.

We argue that lightweight modular staging enables a form of language virtualization, i.e. allows to go from a pure-library embedded language to one that is practically equivalent to a stand-alone implementation with only modest effort.

References

  1. }}G. Bracha and W. R. Cook. Mixin-based inheritance. In OOPSLA/ECOOP, pages 303--311, 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. }}C. Calcagno, E. Moggi, and W. Taha. Closed types as a simple approach to safe imperative multi-stage programming. In U. Montanari, J. D. P. Rolim, and E. Welzl, editors, ICALP, volume 1853 of Lecture Notes in Computer Science, pages 25--36. Springer, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. }}C. Calcagno, W. Taha, L. Huang, and X. Leroy. Implementing multi-stage languages using asts, gensym, and reflection. In F. Pfenning and Y. Smaragdakis, editors, GPCE, volume 2830 of Lecture Notes in Computer Science, pages 57--76. Springer, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. }}J. Carette. Gaussian elimination: A case study in efficient genericity with metaocaml. Sci. Comput. Program., 62 (1): 3--24, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. }}J. Carette and O. Kiselyov. Multi-stage programming with functors and monads: Eliminating abstraction overhead from generic code. In GPCE, 2005, pages 256--274. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. }}J. Carette, O. Kiselyov, and C. chieh Shan. Finally tagless, partially evaluated. In Z. Shao, editor, APLAS, volume 4807 of Lecture Notes in Computer Science, pages 222--238. Springer, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. }}J. Carette, O. Kiselyov, and C. chieh Shan. Finally tagless, partially evaluated: Tagless staged interpreters for simpler typed languages. J. Funct. Program., 19 (5): 509--543, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. }}H. Chafi, Z. DeVito, A. Moors, T. Rompf, A. Sujeeth, P. Hanrahan, M. Odersky, and K. Olukotun. Language Virtualization for Heterogeneous Parallel Computing. In Onward!, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. }}A. Cohen, S. Donadio, M. J. Garzarán, C. A. Herrmann, O. Kiselyov, and D. A. Padua. In search of a program generator to implement generic transformations for high-performance computing. Sci. Comput. Program., 62 (1): 25--46, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. }}K. Czarnecki, J. T. O'Donnell, J. Striegnitz, and W. Taha. Dsl implementation in metaocaml, template haskell, and c++. In Domain-Specific Program Generation, International Seminar, Dagstuhl Castle, Germany, March 23--28, 2003, Revised Papers, volume 3016 of Lecture Notes in Computer Science, 2004, Springer, pages 51--72.Google ScholarGoogle Scholar
  11. }}J. Eckhardt, R. Kaiabachev, E. Pasalic, K. N. Swadi, and W. Taha. Implicitly heterogeneous multi-stage programming. In GPCE, 2005, pages 275--292. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. }}J. Eckhardt, R. Kaiabachev, E. Pasalic, K. N. Swadi, and W. Taha. Implicitly heterogeneous multi-stage programming. New Generation Comput., 25 (3): 305--336, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. }}C. Elliott, S. Finne, and O. de Moor. Compiling embedded languages. In W. Taha, editor, SAIG, volume 1924 of Lecture Notes in Computer Science, pages 9--27. Springer, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. }}C. Elliott, S. Finne, and O. de Moor. Compiling embedded languages. J. Funct. Program., 13 (3): 455--481, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. }}B. Emir, M. Odersky, and J. Williams. Matching objects with patterns. In E. Ernst, editor, ECOOP, volume 4609 of Lecture Notes in Computer Science, pages 273--298. Springer, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. }}M. Fähndrich, M. Carbin, and J. R. Larus. Reflective program generation with patterns. In S. Jarzabek, D. C. Schmidt, and T. L. Veldhuizen, editors, GPCE, pages 275--284. ACM, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. }}C. Flanagan, A. Sabry, B. F. Duba, and M. Felleisen. The essence of compiling with continuations. In PLDI, pages 237--247, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. }}S. E. Ganz, A. Sabry, and W. Taha. Macros as multi-stage computations: Type-safe, generative, binding macros in macroml. In ICFP, pages 74--85, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. }}J. Gillenwater, G. Malecha, C. Salama, A. Y. Zhu, W. Taha, J. Grundy, and J. O'Leary. Synthesizable high level hardware descriptions: using statically typed two-level languages to guarantee verilog synthesizability. In PEPM, 2008, pages 41--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. }}R. Glück and O. de Moor, editors. Proceedings of the 2008 ACM SIGPLAN Symposium on Partial Evaluation and Semantics-based Program Manipulation, PEPM 2008, San Francisco, California, USA, January 7--8, 2008, 2008. ACM.Google ScholarGoogle Scholar
  21. }}R. Glück and M. R. Lowry, editors. Generative Programming and Component Engineering, 4th International Conference, GPCE 2005, Tallinn, Estonia, September 29 - October 1, 2005, Proceedings, volume 3676 of Lecture Notes in Computer Science, 2005. Springer.Google ScholarGoogle ScholarCross RefCross Ref
  22. }}M. Guerrero, E. Pizzi, R. Rosenbaum, K. N. Swadi, and W. Taha. Implementing dsls in metaocaml. In J. M. Vlissides and D. C. Schmidt, editors, OOPSLA Companion, pages 41--42. ACM, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. }}W. L. Harrison and S. N. Kamin. Metacomputation-based compiler architecture. In R. C. Backhouse and J. N. Oliveira, editors, MPC, volume 1837 of Lecture Notes in Computer Science, pages 213--229. Springer, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. }}C. Hofer, K. Ostermann, T. Rendel, and A. Moors. Polymorphic embedding of dsls. In Y. Smaragdakis and J. G. Siek, editors, GPCE, pages 137--148. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. }}S. S. Huang and Y. Smaragdakis. Expressive and safe static reflection with morphj. In R. Gupta and S. P. Amarasinghe, editors, PLDI, pages 79--89. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. }}S. S. Huang, D. Zook, and Y. Smaragdakis. Statically safe program generation with safegen. In GPCE, 2005, pages 309--326. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. }}P. Hudak. Building domain-specific embedded languages. ACM Comput. Surv., 28 (4es): 196, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. }}P. Hudak. Modular domain specific languages and tools. In Proceedings of Fifth International Conference on Software Reuse, pages 134--142, June 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. }}U. Jørring and W. L. Scherlis. Compilers and staging transformations. In POPL, pages 86--96, 1986. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. }}Y. Kameyama, O. Kiselyov, and C. chieh Shan. Closing the stage: from staged code to typed closures. In PEPM, 2008, pages 147--157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. }}Y. Kameyama, O. Kiselyov, and C. chieh Shan. Shifting the stage: staging with delimited control. In PEPM, 2009, pages 111--120. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. }}S. Kamin, L. Clausen, and A. Jarvis. Jumbo: Run-time code generation for java and its applications. In CGO, pages 48--58. IEEE Computer Society, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. }}G. Karsai and E. Visser, editors. Generative Programming and Component Engineering: Third International Conference, GPCE 2004, Vancouver, Canada, October 24--28, 2004. Proceedings, volume 3286 of Lecture Notes in Computer Science, 2004. Springer.Google ScholarGoogle Scholar
  34. }}A. Kennedy. Compiling with continuations, continued. In R. Hinze and N. Ramsey, editors, ICFP, pages 177--190. ACM, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. }}O. Kiselyov and W. Taha. Relating fftw and split-radix. In ICESS, 2004, pages 488--493. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. }}O. Kiselyov, K. N. Swadi, and W. Taha. A methodology for generating verified combinatorial circuits. In G. C. Buttazzo, editor, EMSOFT, pages 249--258. ACM, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. }}D. Leijen and E. Meijer. Domain specific embedded compilers. In DSL, pages 109--122, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. }}C. Lengauer, D. S. Batory, C. Consel, and M. Odersky, editors. Domain-Specific Program Generation, International Seminar, Dagstuhl Castle, Germany, March 23--28, 2003, Revised Papers, volume 3016 of Lecture Notes in Computer Science, 2004. Springer.Google ScholarGoogle Scholar
  39. }}N. Linger and T. Sheard. Binding-time analysis for metaml via type inference and constraint solving. In K. Jensen and A. Podelski, editors, TACAS, volume 2988 of Lecture Notes in Computer Science, pages 266--279. Springer, 2004.Google ScholarGoogle Scholar
  40. }}E. Moggi, W. Taha, Z.-E.-A. Benaissa, and T. Sheard. An idealized metaml: Simpler, and more expressive. In S. D. Swierstra, editor, ESOP, volume 1576 of Lecture Notes in Computer Science, pages 193--207. Springer, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. }}A. Moors, F. Piessens, and M. Odersky. Generics of a higher kind. In G. E. Harris, editor, OOPSLA, pages 423--438. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. }}G. Neverov and P. Roe. Metaphor: A multi-stage, object-oriented programming language. In GPCE, 2004, pages 168--185.Google ScholarGoogle ScholarCross RefCross Ref
  43. }}G. Neverov and P. Roe. Experiences with an object-oriented, multi-stage language. Sci. Comput. Program., 62 (1): 85--94, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. }}M. Odersky and M. Zenger. Scalable component abstractions. In R. E. Johnson and R. P. Gabriel, editors, OOPSLA, pages 41--57. ACM, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. }}M. Odersky and M. Zenger. Independently extensible solutions to the expression problem. In Proc. FOOL 12, Jan. 2005. http://homepages.inf.ed.ac.uk/wadler/foo.Google ScholarGoogle Scholar
  46. }}Y. Oiwa, H. Masuhara, and A. Yonezawa. Dynjava: Type safe dynamic code generation in java. In PPL'01: JSSST Workshop on Programming and Programming Languages, 2001.Google ScholarGoogle Scholar
  47. }}B. C. D. S. Oliveira. Modular visitor components. In S. Drossopoulou, editor, ECOOP, volume 5653 of Lecture Notes in Computer Science, pages 269--293. Springer, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. }}E. Pasalic and N. Linger. Meta-programming with typed object-language representations. In GPCE, 2004, pages 136--167.Google ScholarGoogle Scholar
  49. }}E. Pasalic, W. Taha, and T. Sheard. Tagless staged interpreters for typed languages. In ICFP, pages 218--229, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. }}M. Poletto, W. C. Hsieh, D. R. Engler, and M. F. Kaashoek. 'c and tcc: A language and compiler for dynamic code generation. ACM Trans. Program. Lang. Syst., 21 (2): 324--369, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. }}G. Puebla and G. Vidal, editors. Proceedings of the 2009 ACM SIGPLAN Symposium on Partial Evaluation and Semantics-based Program Manipulation, PEPM 2009, Savannah, GA, USA, January 19--20, 2009, 2009. ACM. Google ScholarGoogle Scholar
  52. }}T. Rendel, K. Ostermann, and C. Hofer. Typed self-representation. In M. Hind and A. Diwan, editors, PLDI, pages 293--303. ACM, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. }}C. Salama, G. Malecha, W. Taha, J. Grundy, and J. O'Leary. Static consistency checking for verilog wire interconnects: using dependent types to check the sanity of verilog descriptions. In PEPM, 2009, pages 121--130. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. }}U. P. Schultz, J. L. Lawall, and C. Consel. Automatic program specialization for java. ACM Trans. Program. Lang. Syst., 25 (4): 452--499, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. }}S. Seefried, M. M. T. Chakravarty, and G. Keller. Optimising embedded dsls using template haskell. In GPCE, 2004, pages 186--205.Google ScholarGoogle ScholarCross RefCross Ref
  56. }}P. Sestoft. Runtime code generation with jvm and clr. http://www.itu.dk/people/sestoft/rtcg/rtcg.pdf, 2002.Google ScholarGoogle Scholar
  57. }}T. Sheard. Accomplishments and research challenges in meta-programming. In W. Taha, editor, SAIG, volume 2196 of Lecture Notes in Computer Science, pages 2--44. Springer, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. }}T. Sheard and S. L. P. Jones. Template meta-programming for haskell. SIGPLAN Notices, 37 (12): 60--75, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. }}T. Sheard and E. Pasalic. Meta-programming with built-in type equality. Electr. Notes Theor. Comput. Sci., 199: 49--65, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. }}T. Sheard, Z.-E.-A. Benaissa, and E. Pasalic. Dsl implementation using staging and monads. In DSL, pages 81--94, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. }}J. G. Siek. General purpose languages should be metalanguages. In J. P. Gallagher and J. Voigtländer, editors, PEPM, pages 3--4. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. }}J. G. Siek and W. Taha. A semantic analysis of c++ templates. In D. Thomas, editor, ECOOP, volume 4067 of Lecture Notes in Computer Science, pages 304--327. Springer, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. }}F. Smith, D. Grossman, J. G. Morrisett, L. Hornof, and T. Jim. Compiling for template-based run-time code generation. J. Funct. Program., 13 (3): 677--708, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. }}K. N. Swadi, W. Taha, O. Kiselyov, and E. Pasalic. A monadic approach for avoiding code duplication when staging memoized functions. In J. Hatcliff and F. Tip, editors, PEPM, pages 160--169. ACM, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. }}W. Taha. A sound reduction semantics for untyped cbn multi-stage computation. or, the theory of metaml is non-trivial (extended abstract). In PEPM, pages 34--43, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. }}W. Taha. A gentle introduction to multi-stage programming. In Domain-Specific Program Generation, International Seminar, Dagstuhl Castle, Germany, March 23--28, 2003, Revised Papers, volume 3016 of Lecture Notes in Computer Science, 2004, Springer, pages 30--50.Google ScholarGoogle Scholar
  67. }}W. Taha. Resource-aware programming. In ICESS, 2004, pages 38--43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. }}W. Taha. A gentle introduction to multi-stage programming, part ii. In R. Lämmel, J. Visser, and J. Saraiva, editors, GTTSE, volume 5235 of Lecture Notes in Computer Science, pages 260--290. Springer, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. }}W. Taha and M. F. Nielsen. Environment classifiers. In POPL, pages 26--37, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. }}W. Taha and T. Sheard. Multi-stage programming. In ICFP, page 321, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. }}W. Taha and T. Sheard. Multi-stage programming with explicit annotations. In PEPM, pages 203--217, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. }}W. Taha and T. Sheard. Metaml and multi-stage programming with explicit annotations. Theor. Comput. Sci., 248 (1--2): 211--242, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. }}W. Taha, Z.-E.-A. Benaissa, and T. Sheard. Multi-stage programming: Axiomatization and type safety. In K. G. Larsen, S. Skyum, and G. Winskel, editors, ICALP, volume 1443 of Lecture Notes in Computer Science, pages 918--929. Springer, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. }}M. Torgersen. The expression problem revisited. In M. Odersky, editor, ECOOP, volume 3086 of Lecture Notes in Computer Science, pages 123--143. Springer, 2004.Google ScholarGoogle Scholar
  75. }}D. Vandevoorde and N. M. Josuttis. C++ Templates: The Complete Guide. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. }}T. L. Veldhuizen. Active Libraries and Universal Languages. PhD thesis, Indiana University Computer Science, May 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. }}P. Wadler. The expression problem. Posted on the Java Genericity mailing list, 1998.Google ScholarGoogle Scholar
  78. }}E. Westbrook, M. Ricken, J. Inoue, Y. Yao, T. Abdelatif, and W. Taha. Mint: Java multi-stage programming using weak separability. In PLDI, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. }}Z. Wu, C. Chen, M. Guo, and J. Bu, editors. Embedded Software and Systems, First International Conference, ICESS 2004, Hangzhou, China, December 9--10, 2004, Revised Selected Papers, volume 3605 of Lecture Notes in Computer Science, 2005. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. }}D. Zook, S. S. Huang, and Y. Smaragdakis. Generating aspectj programs with meta-aspectj. In GPCE, 2004, pages 1--18..Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Lightweight modular staging: a pragmatic approach to runtime code generation and compiled DSLs

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 46, Issue 2
      GPCE '10
      Febuary 2011
      185 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/1942788
      Issue’s Table of Contents
      • cover image ACM Conferences
        GPCE '10: Proceedings of the ninth international conference on Generative programming and component engineering
        October 2010
        198 pages
        ISBN:9781450301541
        DOI:10.1145/1868294
        • General Chair:
        • Eelco Visser,
        • Program Chair:
        • Jaakko Järvi

      Copyright © 2010 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 10 October 2010

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!