Abstract
Syntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation algorithm. They hamper evolution, maintainability, and compositionality of syntax definitions. The pureness and declarativity of syntax definitions is lost. We analyze how these problems arise for different aspects of syntax definitions, discuss their consequences for language engineers, and show how the pure and declarative nature of syntax definitions can be regained.
- }}A. V. Aho, S. C. Johnson, and J. D. Ullman. Deterministic parsing of ambiguous grammars. Commun. ACM, 18(8):441--452, 1975. Google Scholar
Digital Library
- }}A. V. Aho and T. G. Peterson. A minimum distance error-correcting parser for context-free languages. SIAM J. Comput., 1(4):305--312, 1972.Google Scholar
Cross Ref
- }}A. Birman and J. D. Ullman. Parsing algorithms with backtrack. In Conference Record of 1970 Eleventh Annual Symposium on Switching and Automata Theory, 28-30 October 1970, Santa Monica, California, USA, pages 153--174. IEEE, 1970. Google Scholar
Digital Library
- }}A. Borghi, V. David, and A. Demaille. C-Transformers: a framework to write C program transformations. ACM Crossroads, 12(3):3, 2005. Google Scholar
Digital Library
- }}E. Bouwers, M. Bravenboer, and E. Visser. Grammar engineering support for precedence rule recovery and compatibility checking. In A. Sloane and A. Johnstone, editors, Seventh Workshop on Language Descriptions, Tools, and Applications (LDTA 2007), volume 203 of Electronic Notes in Theoretical Computer Science, pages 85--101, Braga, Portugal, March 2008. Elsevier. Google Scholar
Digital Library
- }}M. G. J. van den Brand, H. de Jong, P. Klint, and P. Olivier. Efficient annotated terms. Software, Practice & Experience, 30(3):259--291, 2000. Google Scholar
Digital Library
- }}M. G. J. van den Brand, J. Scheerder, J. Vinju, and E. Visser. Disambiguation filters for scannerless generalized LR parsers. In N. Horspool, editor, Compiler Construction (CC 2002), volume 2304 of Lecture Notes in Computer Science, pages 143--158, Grenoble, France, April 2002. Springer-Verlag. Google Scholar
Digital Library
- }}M. G. J. van den Brand and E. Visser. Generation of formatters for context-free languages. TOSEM, 5(1):1--41, January 1996. Google Scholar
Digital Library
- }}M. Bravenboer, E. Dolstra, and E. Visser. Preventing injection attacks with syntax embeddings. In C. Consel and J. L. Lawall, editors, Generative Programming and Component Engineering, 6th International Conference, GPCE 2007, pages 3--12, Salzburg, Austria, 2007. ACM. Google Scholar
Digital Library
- }}M.Bravenboer,EricTanter,andE.Visser.Declarative,formal, and extensible syntax definition for AspectJ. In P. L. Tarr and W. R. Cook, editors, Proceedings of the 21th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2006, October 22-26, 2006, Portland, Oregon, USA, pages 209--228. ACM, 2006. Google Scholar
Digital Library
- }}D. G. Cantor. On the ambiguity problem of backus systems. Journal of the ACM, 9(4):477--479, 1962. Google Scholar
Digital Library
- }}P. Charles. A practical method for constructing efficient LALR(K) parsers with automatic error recovery. PhD thesis, New York University, 1991. Google Scholar
Digital Library
- }}N. Chomsky. Syntactic Structures. Mouton de Gruyter, 1957.Google Scholar
- }}J. Cocke. Programming languages and their compilers: Preliminary notes. Courant Institute of Mathematical Sciences, New York University, 1969. Google Scholar
Digital Library
- }}J. R. Cordy, C. D. Halpern-Hamu, and E. Promislow. TXL: a rapid prototyping system for programming language dialects. In Conf. on Comp. Languages, pages 280--285. IEEE, 1988.Google Scholar
Cross Ref
- }}M. de Jonge. A pretty-printer for every occasion. In The International Symposium on Constructing Software Engineering Tools (CoSET2000), pages 68--77. University of Wollongong, Australia, 2000.Google Scholar
- }}M. de Jonge, E. Nilsson-Nyman, L. C. L. Kats, and E. Visser. Natural and flexible error recovery for generated parsers. In M. van den Brand, D. Gasevic, and J. Gray, editors, Second International Conference, SLE 2009, Denver, CO, USA, October 5-6, 2009, Revised Selected Papers, Lecture Notes in Computer Science. Springer, 2010. Google Scholar
Digital Library
- }}F. L. DeRemer. Practical translation for LR(k) languages. PhD thesis, MIT, Cambridge, MA, USA, September 1969.Google Scholar
- }}J. Earley. An efficient context-free parsing algorithm. Commun. ACM, 13(2):94--102, 1970. Google Scholar
Digital Library
- }}J.-M. Favre. Languages evolve too! Changing the software time scale. In 8th International Workshop on Principles of Software Evolution (IWPSE 2005), 5-7 September 2005, Lisbon, Portugal, pages 33--44. IEEE Computer Society, 2005. Google Scholar
Digital Library
- }}R. Filman and D. Friedman. Aspect-oriented programming is quantification and obliviousness. In Workshop on Advanced separation of Concerns, 2000. Google Scholar
Digital Library
- }}C. Fischer and R. LeBlanc. Crafting a compiler. Benjamin/Cummings Menlo Park, California, USA, 1988. Google Scholar
Digital Library
- }}B. Ford. Packrat parsing: simple, powerful, lazy, linear time, functional pearl. In Proceedings of the seventh ACM SIGPLAN international conference on Functional Programming (ICFP 2002), pages 36--47, 2002. Google Scholar
Digital Library
- }}B. Ford. Parsing expression grammars: a recognition-based syntactic foundation.In N. D. Jones and X. Leroy, editors, Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2004, Venice, Italy, January 14-16, 2004, pages 111--122. ACM, 2004. Google Scholar
Digital Library
- }}S. Ginsburg and J. Ullian. Ambiguity in context free languages. J. ACM, 13(1):62--89, 1966. Google Scholar
Digital Library
- }}S. Gorn. Detection of generative ambiguities in context-free mechanical languages. J. ACM, 10(2):196--208, 1963. Google Scholar
Digital Library
- }}S. L. Graham, C. B. Haley, and W. N. Joy. Practical LR error recovery. In Proceedings of the 1979 SIGPLAN Symposium on Compiler Construction, Denver, Colorado, USA, August 6-10, 1979, pages 168--175. ACM, 1979. Google Scholar
Digital Library
- }}J. Heering, P. R. H. Hendriks, P. Klint, and J. Rekers. The syntax definition formalism SDF - reference manual. SIGPLAN Notices, 24(11):43--75, 1989. Google Scholar
Digital Library
- }}S. C. Johnson. YACC - yet another compiler-compiler. Technical Report CS-32, AT & T Bell Laboratories, Murray Hill, N.J., 1975.Google Scholar
- }}A. Johnstone, E. Scott, and G. Economopoulos. Evaluating GLR parsing algorithms. Science of Computer Programming, 61(3):228--244, 2006. Google Scholar
Digital Library
- }}T. Kasami. An efficient recognition and syntax analysis algorithm for context free languages. Science Report AF CRL-65-758, Air Force Cambridge Research Laboratory, Bedford, Mass., USA, 1965.Google Scholar
- }}L. C. L. Kats, M. de Jonge, E. Nilsson-Nyman, and E. Visser. Providing rapid feedback in generated modular language environments. Adding error recovery to scannerless generalized-LR parsing. In S. Arora and G. T. Leavens, editors, Proceedings of the 24th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2009, October 25--29, 2009, Orlando, Florida, USA., pages 445--464, 2009. Google Scholar
Digital Library
- }}L. C. L. Kats and E. Visser. The Spoofax language workbench. Rules for declarative specification of languages and IDEs. In M. Rinard, editor, Proceedings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2010, October 17-21, 2010, Reno, NV, USA, 2010. Google Scholar
Digital Library
- }}P. Klint, R. Lammel, and C. Verhoef. Toward an engineering discipline for grammarware. ACM Transactions on Software Engineering Methodology, 14(3):331--380, 2005. Google Scholar
Digital Library
- }}P. Klint and E. Visser. Using filters for the disambiguation of context-free grammars. In Proc. ASMICS Workshop on Parsing Theory, pages 1--20, Milano, Italy, October 1994. Tech. Rep. 126-1994, Dipartimento di Scienze dell'Informazione, Universita di Milano.Google Scholar
- }}D. E. Knuth. On the translation of languages from left to right. Information and control, 8(6):607--639, 1965.Google Scholar
Cross Ref
- }}P. M. Lewis II and R. E. Stearns. Syntax-directed transduction. Journal of the ACM, 15(3):465--488, 1968. Google Scholar
Digital Library
- }}B. A. Malloy, J. F. Power, and J. T. Waldron. Applying software engineering techniques to parser design: the development of a C# parser. In SAICSIT '02: Proceedings of the 2002 annual research conference of the South African institute of computer scientists and information technologists on Enablement through technology, pages 75--82, Port Elizabeth, Republic of South Africa, 2002. South African Institute for Computer Scientists and Information Technologists. Google Scholar
Digital Library
- }}P. M. Maurer. Generating test data with enhanced context-free grammars. IEEE Software, 7(4):50--55, 1990. Google Scholar
Digital Library
- }}Panini Astadhyay1. Otto Bohtlingk, editor, Konig, 1839--1840.Google Scholar
- }}Panini Astadhyay1. Benares, 1896. Translated by Sr1sa Chandra Vasu.Google Scholar
- }}T. J. Parr. ANTLR Parser Generator. http://www.antlr.org/.Google Scholar
- }}T. J. Parr and R. W. Quong. LL and LR translators need k>1 lookahead. SIGPLAN Not., 31(2):27--34, 1996. Google Scholar
Digital Library
- }}J. Rekers. Parser Generation for Interactive Environments. PhD thesis, University of Amsterdam, Amsterdam, The Netherlands, January 1992.Google Scholar
- }}D. Salomon and G. Cormack. The disambiguation and scannerless parsing of complete character-level grammars for programming languages. Technical Report 95/06, Department of Computer Science, University of Manitoba, Winnipeg, Canada, 1995.Google Scholar
- }}D. J. Salomon and G. V. Cormack. Scannerless NSLR(1) parsing of programming languages. SIGPLAN Not., 24(7):170--178, 1989. Google Scholar
Digital Library
- }}E. Scott and A. Johnstone. GLL parsing. In Workshop on Language Descriptions, Tools and Applications (LDTA'09), 2009.Google Scholar
- }}The Spoofax project. http://www.spoofax.org/.Google Scholar
- }}M. Tomita. Efficient Parsing for Natural Language: A Fast Algorithm for Practical Systems, volume 14. Kluwer Academic Publishers, 1988. Google Scholar
Digital Library
- }}E. Visser. A case study in optimizing parsing schemata by disambiguation filters. In IWPT, pages 210--224, 1997.Google Scholar
- }}E. Visser. Scannerless generalized-LR parsing. Technical Report P9707, Programming Research Group, University of Amsterdam, July 1997.Google Scholar
- }}E. Visser. Syntax Definition for Language Prototyping. PhD thesis, University of Amsterdam, September 1997.Google Scholar
- }}E. Visser. Meta-programming with concrete object syntax. In D. S. Batory, C. Consel, and W. Taha, editors, Generative Programming and Component Engineering, ACM SIGPLAN/SIGSOFT Conference, GPCE 2002, Pittsburgh, PA, USA, October 6-8, 2002, Proceedings, volume 2487 of Lecture Notes in Computer Science, pages 299--315. Springer, 2002. Google Scholar
Digital Library
- }}D. Younger. Recognition and parsing of context-free languages in time n3. Information and control, 10(2):189--208, 1967.Google Scholar
Cross Ref
Index Terms
Pure and declarative syntax definition: paradise lost and regained
Recommendations
Integrated language definition testing: enabling test-driven language development
OOPSLA '11The reliability of compilers, interpreters, and development environments for programming languages is essential for effective software development and maintenance. They are often tested only as an afterthought. Languages with a smaller scope, such as ...
Pure and declarative syntax definition: paradise lost and regained
OOPSLA '10: Proceedings of the ACM international conference on Object oriented programming systems languages and applicationsSyntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation ...
The spoofax language workbench: rules for declarative specification of languages and IDEs
OOPSLA '10: Proceedings of the ACM international conference on Object oriented programming systems languages and applicationsSpoofax is a language workbench for efficient, agile development of textual domain-specific languages with state-of-the-art IDE support. Spoofax integrates language processing techniques for parser generation, meta-programming, and IDE development into ...







Comments