SugarJ: library-based syntactic language extensibility

ABSTRACT

Existing approaches to extend a programming language with syntactic sugar often leave a bitter taste, because they cannot be used with the same ease as the main extension mechanism of the programming language - libraries. Sugar libraries are a novel approach for syntactically extending a programming language within the language. A sugar library is like an ordinary library, but can, in addition, export syntactic sugar for using the library. Sugar libraries maintain the composability and scoping properties of ordinary libraries and are hence particularly well-suited for embedding a multitude of domain-specific languages into a host language. They also inherit self-applicability from libraries, which means that sugar libraries can provide syntactic extensions for the definition of other sugar libraries. To demonstrate the expressiveness and applicability of sugar libraries, we have developed SugarJ, a language on top of Java, SDF and Stratego, which supports syntactic extensibility. SugarJ employs a novel incremental parsing technique, which allows changing the syntax within a source file. We demonstrate SugarJ by five language extensions, including embeddings of XML and closures in Java, all available as sugar libraries. We illustrate the utility of self-applicability by embedding XML Schema, a metalanguage to define XML languages.

References

1. E. Allen, R. Culpepper, J. D. Nielsen, J. Rafkind, and S. Ryu. Growing a syntax. In Proceedings of Workshop on Foundations of Object-Oriented Languages, 2009. Available at http://www.cs.cmu.edu/aldrich/FOOL09/allen.pdf.
2. T. Aoto, J. Yoshida, and Y. Toyama. Proving confluence of term rewriting systems automatically. In Proceedings of Conference on Rewriting Techniques and Applications (RTA), volume 5595 of LNCS, pages 93--102. Springer, 2009.
3. R. Axelsson, K. Heljanko, and M. Lange. Analyzing context-free grammars using an incremental SAT solver. In Proceedings of International Colloquium on Automata, Languages and Programming (ICALP), volume 5125 of LNCS, pages 410--422. Springer, 2008.
4. J. Bachrach and K. Playford. The Java syntactic extender (JSE). In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 31--42. ACM, 2001.
5. D. Batory, B. Lofaso, and Y. Smaragdakis. JTS: Tools for implementing domain-specific languages. In Proceedings of International Conference on Software Reuse (ICSR), pages 143--153. IEEE, 1998.
6. C. Brabrand and M. I. Schwartzbach. Growing languages with metamorphic syntax macros. In Proceedings of Workshop on Partial Evaluation and Program Manipulation (PEPM), pages 31--40. ACM, 2002.
7. G. Bracha. Pluggable type systems. In OOPSLA Workshop on Revival of Dynamic Languages, 2004. Available at http://bracha.org/pluggableTypesPosition.pdf.
8. M. Bravenboer, E. Dolstra, and E. Visser. Preventing injection attacks with syntax embeddings. Science of Computer Programming, 75(7):473--495, 2010.
9. M. Bravenboer and E. Visser. Concrete syntax for objects: Domain-specific language embedding and assimilation without restrictions. In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 365--383. ACM, 2004.
10. M. Calder, M. Kolberg, E. H. Magill, and S. Reiff-Marganiec. Feature interaction: A critical review and considered forecast. Computer Networks, 41(1):115--141, 2003.
11. L. Cardelli. Program fragments, linking, and modularization. In Proceedings of Symposium on Principles of Programming Languages (POPL), pages 266--277. ACM, 1997.
12. W. Clinger and J. Rees. Macros that work. In Proceedings of Symposium on Principles of Programming Languages (POPL), pages 155--162. ACM, 1991.
13. S. Dmitriev. Language oriented programming: The next programming paradigm. Available at http://www.jetbrains.com/mps/docs/Language_Oriented_Programming.pdf, 2004.
14. T. Ekman and G. Hedin. The JastAdd extensible Java compiler. In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 1--18. ACM, 2007.
15. S. Erdweg, L. C. L. Kats, T. Rendel, C. K\"astner, K. Ostermann, and E. Visser. Growing a language environment with editor libraries. In Proceedings of Conference on Generative Programming and Component Engineering (GPCE). ACM, 2011.
16. S. Erdweg and K. Ostermann. Featherweight TeX and parser correctness. In Proceedings of Conference on Software Language Engineering (SLE), volume 6563 of LNCS, pages 397--416. Springer, 2010.
17. M. Flatt. Composable and compilable macros: You want it when? In Proceedings of International Conference on Functional Programming (ICFP), pages 72--83. ACM, 2002.
18. M. Flatt, E. Barzilay, and R. B. Findler. Scribble: Closing the book on ad hoc documentation tools. In Proceedings of International Conference on Functional Programming (ICFP), pages 109--120. ACM, 2009.
19. M. Fowler. Language workbenches: The killer-app for domain specific languages? Available at http://martinfowler.com/articles/languageWorkbench.html, 2005.
20. N. Gafter and P. von der Ahé. Closures for Java. Available at http://javac.info/closures-v06a.html.
21. J. Heering, P. R. H. Hendriks, P. Klint, and J. Rekers. The syntax definition formalism SDF -- reference manual. SIGPLAN Notices, 24(11):43--75, 1989.
22. C. Hofer and K. Ostermann. Modular domain-specific language components in Scala. In Proceedings of Conference on Generative Programming and Component Engineering (GPCE), pages 83--92. ACM, 2010.
23. C. Hofer, K. Ostermann, T. Rendel, and A. Moors. Polymorphic embedding of DSLs. In Proceedings of Conference on Generative Programming and Component Engineering (GPCE), pages 137--148. ACM, 2008.
24. P. Hudak. Modular domain specific languages and tools. In Proceedings of International Conference on Software Reuse (ICSR), pages 134--142. IEEE, 1998.
25. L. C. L. Kats and E. Visser. The Spoofax language workbench: Rules for declarative specification of languages and IDEs. In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 444--463. ACM, 2010.
26. O. Kiselyov. Macros that compose: Systematic macro programming. In Proceedings of Conference on Generative Programming and Component Engineering (GPCE), volume 2487 of LNCS, pages 202--217. Springer, 2002.
27. E. Kohlbecker, D. P. Friedman, M. Felleisen, and B. Duba. Hygienic macro expansion. In Proceedings of Conference on LISP and Functional Programming (LFP), pages 151--161. ACM, 1986.
28. S. Krishnamurthi. Educational pearl: Automata via macros. Journal of Functional Programming, 16(3):253--267, 2006.
29. B. M. Leavenworth. Syntax macros and extended translation. Communications of the ACM, 9:790--793, 1966.
30. T. Mens, G. Taentzer, and O. Runge. Detecting structural refactoring conflicts using critical pair analysis. Electronic Notes in Theoretical Computer Science, 127(3):113--128, 2005.
31. M. Mernik, J. Heering, and A. M. Sloane. When and how to develop domain-specific languages. ACM Computing Surveys, 37:316--344, 2005.
32. A. Møller and M. I. Schwartzbach. An Introduction to XML and Web Technologies. Addison-Wesley, 2006.
33. N. Nystrom, M. R. Clarkson, and A. C. Myers. Polyglot: An extensible compiler framework for Java. In Proceedings of Conference on Compiler Construction (CC), volume 2622 of LNCS, pages 138--152. Springer, 2003.
34. M. Odersky. The Scala language specication, version 2.8. Available at http://www.scala-lang.org/docu/files/ScalaReference.pdf., 2010.
35. B. C. Oliveira. Modular visitor components. In Proceedings of European Conference on Object-Oriented Programming (ECOOP), volume 5653 of LNCS, pages 269--293. Springer, 2009.
36. L. Renggli, M. Denker, and O. Nierstrasz. Language boxes: Bending the host language with modular language changes. In Proceedings of Conference on Software Language Engineering (SLE), volume 5969 of LNCS, pages 274--293. Springer, 2009.
37. L. Renggli, T. Gırba, and O. Nierstrasz. Embedding languages without breaking tools. In Proceedings of European Conference on Object-Oriented Programming (ECOOP), volume 6183 of LNCS, pages 380--404. Springer, 2010.
38. S. Schmitz. Conservative ambiguity detection in context-free grammars. In Proceedings of International Colloquium on Automata, Languages and Programming (ICALP), volume 4596 of LNCS, pages 692--703. Springer, 2007.
39. C. Seaton. A programming language where the syntax and semantics are mutable at runtime. Master's thesis, University of Bristol, 2007.
40. J. N. Shutt. Recursive adaptable grammars. Master's thesis, Worcester Polytechnic Institute, 1993.
41. C. Simonyi. The death of computer languages, the birth of intentional programming. In NATO Science Committee Conference, 1995.
42. M. Tatsubori, S. Chiba, M.-O. Killijian, and K. Itano. OpenJava: A class-based macro system for Java. In Proceedings of Workshop on Reflection and Software Engineering, volume 1826 of LNCS, pages 117--133. Springer, 2000.
43. S. Tobin-Hochstadt, V. St-Amour, R. Culpepper, M. Flatt, and M. Felleisen. Languages as libraries. In Proceedings of Conference on Programming Language Design and Implementation (PLDI). ACM, 2011.
44. L. Tratt. Domain specific language implementation via compile-time meta-programming. Transactions on Programming Languages and Systems (TOPLAS), 30(6):1--40, 2008.
45. M. van den Brand, J. Scheerder, J. J. Vinju, and E. Visser. Disambiguation filters for scannerless generalized LR parsers. In Proceedings of Conference on Compiler Construction (CC), volume 2304 of LNCS, pages 143--158. Springer, 2002.
46. E. Van Wyk, D. Bodin, J. Gao, and L. Krishnan. Silver: An extensible attribute grammar system. Science of Computer Programming, 75(1--2):39--54, 2010.
47. E. Van Wyk, L. Krishnan, D. Bodin, and A. Schwerdfeger. Attribute grammar-based language extensions for Java. In Proceedings of European Conference on Object-Oriented Programming (ECOOP), volume 4609 of LNCS, pages 575--599. Springer, 2007.
48. E. Visser. Stratego: A language for program transformation based on rewriting strategies. In Proceedings of Conference on Rewriting Techniques and Applications (RTA), volume 2051 of LNCS, pages 357--362. Springer, 2001.
49. E. Visser. Meta-programming with concrete object syntax. In Proceedings of Conference on Generative Programming and Component Engineering (GPCE), volume 2487 of LNCS, pages 299--315. Springer, 2002.
50. W3C XML Schema Working Group. XML schema part 0: Primer second edition. Available at http://www.w3.org/TR/xmlschema-0, 2004.
51. W3C XML Working Group. Extensible markup language (XML) 1.0 (fifth edition). Available at http://www.w3.org/TR/xml, 2008.
52. M. P. Ward. Language-oriented programming. Software -- Concepts and Tools, 15:147--161, 1995.
53. D. Weise and R. F. Crew. Programmable syntax macros. In Proceedings of Conference on Programming Language Design and Implementation (PLDI), pages 156--165. ACM, 1993.

Index Terms

• Software and its engineering

  • Software creation and management

    • Software development techniques

      • Reusability

  • Software notations and tools

    • General programming languages

      • Language types

        • Extensible languages

    • Software libraries and repositories