skip to main content
research-article

Pure and declarative syntax definition: paradise lost and regained

Published:17 October 2010Publication History
Skip Abstract Section

Abstract

Syntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation algorithm. They hamper evolution, maintainability, and compositionality of syntax definitions. The pureness and declarativity of syntax definitions is lost. We analyze how these problems arise for different aspects of syntax definitions, discuss their consequences for language engineers, and show how the pure and declarative nature of syntax definitions can be regained.

References

  1. }}A. V. Aho, S. C. Johnson, and J. D. Ullman. Deterministic parsing of ambiguous grammars. Commun. ACM, 18(8):441--452, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. }}A. V. Aho and T. G. Peterson. A minimum distance error-correcting parser for context-free languages. SIAM J. Comput., 1(4):305--312, 1972.Google ScholarGoogle ScholarCross RefCross Ref
  3. }}A. Birman and J. D. Ullman. Parsing algorithms with backtrack. In Conference Record of 1970 Eleventh Annual Symposium on Switching and Automata Theory, 28-30 October 1970, Santa Monica, California, USA, pages 153--174. IEEE, 1970. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. }}A. Borghi, V. David, and A. Demaille. C-Transformers: a framework to write C program transformations. ACM Crossroads, 12(3):3, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. }}E. Bouwers, M. Bravenboer, and E. Visser. Grammar engineering support for precedence rule recovery and compatibility checking. In A. Sloane and A. Johnstone, editors, Seventh Workshop on Language Descriptions, Tools, and Applications (LDTA 2007), volume 203 of Electronic Notes in Theoretical Computer Science, pages 85--101, Braga, Portugal, March 2008. Elsevier. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. }}M. G. J. van den Brand, H. de Jong, P. Klint, and P. Olivier. Efficient annotated terms. Software, Practice & Experience, 30(3):259--291, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. }}M. G. J. van den Brand, J. Scheerder, J. Vinju, and E. Visser. Disambiguation filters for scannerless generalized LR parsers. In N. Horspool, editor, Compiler Construction (CC 2002), volume 2304 of Lecture Notes in Computer Science, pages 143--158, Grenoble, France, April 2002. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. }}M. G. J. van den Brand and E. Visser. Generation of formatters for context-free languages. TOSEM, 5(1):1--41, January 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. }}M. Bravenboer, E. Dolstra, and E. Visser. Preventing injection attacks with syntax embeddings. In C. Consel and J. L. Lawall, editors, Generative Programming and Component Engineering, 6th International Conference, GPCE 2007, pages 3--12, Salzburg, Austria, 2007. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. }}M.Bravenboer,EricTanter,andE.Visser.Declarative,formal, and extensible syntax definition for AspectJ. In P. L. Tarr and W. R. Cook, editors, Proceedings of the 21th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2006, October 22-26, 2006, Portland, Oregon, USA, pages 209--228. ACM, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. }}D. G. Cantor. On the ambiguity problem of backus systems. Journal of the ACM, 9(4):477--479, 1962. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. }}P. Charles. A practical method for constructing efficient LALR(K) parsers with automatic error recovery. PhD thesis, New York University, 1991. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. }}N. Chomsky. Syntactic Structures. Mouton de Gruyter, 1957.Google ScholarGoogle Scholar
  14. }}J. Cocke. Programming languages and their compilers: Preliminary notes. Courant Institute of Mathematical Sciences, New York University, 1969. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. }}J. R. Cordy, C. D. Halpern-Hamu, and E. Promislow. TXL: a rapid prototyping system for programming language dialects. In Conf. on Comp. Languages, pages 280--285. IEEE, 1988.Google ScholarGoogle ScholarCross RefCross Ref
  16. }}M. de Jonge. A pretty-printer for every occasion. In The International Symposium on Constructing Software Engineering Tools (CoSET2000), pages 68--77. University of Wollongong, Australia, 2000.Google ScholarGoogle Scholar
  17. }}M. de Jonge, E. Nilsson-Nyman, L. C. L. Kats, and E. Visser. Natural and flexible error recovery for generated parsers. In M. van den Brand, D. Gasevic, and J. Gray, editors, Second International Conference, SLE 2009, Denver, CO, USA, October 5-6, 2009, Revised Selected Papers, Lecture Notes in Computer Science. Springer, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. }}F. L. DeRemer. Practical translation for LR(k) languages. PhD thesis, MIT, Cambridge, MA, USA, September 1969.Google ScholarGoogle Scholar
  19. }}J. Earley. An efficient context-free parsing algorithm. Commun. ACM, 13(2):94--102, 1970. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. }}J.-M. Favre. Languages evolve too! Changing the software time scale. In 8th International Workshop on Principles of Software Evolution (IWPSE 2005), 5-7 September 2005, Lisbon, Portugal, pages 33--44. IEEE Computer Society, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. }}R. Filman and D. Friedman. Aspect-oriented programming is quantification and obliviousness. In Workshop on Advanced separation of Concerns, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. }}C. Fischer and R. LeBlanc. Crafting a compiler. Benjamin/Cummings Menlo Park, California, USA, 1988. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. }}B. Ford. Packrat parsing: simple, powerful, lazy, linear time, functional pearl. In Proceedings of the seventh ACM SIGPLAN international conference on Functional Programming (ICFP 2002), pages 36--47, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. }}B. Ford. Parsing expression grammars: a recognition-based syntactic foundation.In N. D. Jones and X. Leroy, editors, Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2004, Venice, Italy, January 14-16, 2004, pages 111--122. ACM, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. }}S. Ginsburg and J. Ullian. Ambiguity in context free languages. J. ACM, 13(1):62--89, 1966. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. }}S. Gorn. Detection of generative ambiguities in context-free mechanical languages. J. ACM, 10(2):196--208, 1963. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. }}S. L. Graham, C. B. Haley, and W. N. Joy. Practical LR error recovery. In Proceedings of the 1979 SIGPLAN Symposium on Compiler Construction, Denver, Colorado, USA, August 6-10, 1979, pages 168--175. ACM, 1979. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. }}J. Heering, P. R. H. Hendriks, P. Klint, and J. Rekers. The syntax definition formalism SDF - reference manual. SIGPLAN Notices, 24(11):43--75, 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. }}S. C. Johnson. YACC - yet another compiler-compiler. Technical Report CS-32, AT & T Bell Laboratories, Murray Hill, N.J., 1975.Google ScholarGoogle Scholar
  30. }}A. Johnstone, E. Scott, and G. Economopoulos. Evaluating GLR parsing algorithms. Science of Computer Programming, 61(3):228--244, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. }}T. Kasami. An efficient recognition and syntax analysis algorithm for context free languages. Science Report AF CRL-65-758, Air Force Cambridge Research Laboratory, Bedford, Mass., USA, 1965.Google ScholarGoogle Scholar
  32. }}L. C. L. Kats, M. de Jonge, E. Nilsson-Nyman, and E. Visser. Providing rapid feedback in generated modular language environments. Adding error recovery to scannerless generalized-LR parsing. In S. Arora and G. T. Leavens, editors, Proceedings of the 24th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2009, October 25--29, 2009, Orlando, Florida, USA., pages 445--464, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. }}L. C. L. Kats and E. Visser. The Spoofax language workbench. Rules for declarative specification of languages and IDEs. In M. Rinard, editor, Proceedings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2010, October 17-21, 2010, Reno, NV, USA, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. }}P. Klint, R. Lammel, and C. Verhoef. Toward an engineering discipline for grammarware. ACM Transactions on Software Engineering Methodology, 14(3):331--380, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. }}P. Klint and E. Visser. Using filters for the disambiguation of context-free grammars. In Proc. ASMICS Workshop on Parsing Theory, pages 1--20, Milano, Italy, October 1994. Tech. Rep. 126-1994, Dipartimento di Scienze dell'Informazione, Universita di Milano.Google ScholarGoogle Scholar
  36. }}D. E. Knuth. On the translation of languages from left to right. Information and control, 8(6):607--639, 1965.Google ScholarGoogle ScholarCross RefCross Ref
  37. }}P. M. Lewis II and R. E. Stearns. Syntax-directed transduction. Journal of the ACM, 15(3):465--488, 1968. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. }}B. A. Malloy, J. F. Power, and J. T. Waldron. Applying software engineering techniques to parser design: the development of a C# parser. In SAICSIT '02: Proceedings of the 2002 annual research conference of the South African institute of computer scientists and information technologists on Enablement through technology, pages 75--82, Port Elizabeth, Republic of South Africa, 2002. South African Institute for Computer Scientists and Information Technologists. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. }}P. M. Maurer. Generating test data with enhanced context-free grammars. IEEE Software, 7(4):50--55, 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. }}Panini Astadhyay1. Otto Bohtlingk, editor, Konig, 1839--1840.Google ScholarGoogle Scholar
  41. }}Panini Astadhyay1. Benares, 1896. Translated by Sr1sa Chandra Vasu.Google ScholarGoogle Scholar
  42. }}T. J. Parr. ANTLR Parser Generator. http://www.antlr.org/.Google ScholarGoogle Scholar
  43. }}T. J. Parr and R. W. Quong. LL and LR translators need k>1 lookahead. SIGPLAN Not., 31(2):27--34, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. }}J. Rekers. Parser Generation for Interactive Environments. PhD thesis, University of Amsterdam, Amsterdam, The Netherlands, January 1992.Google ScholarGoogle Scholar
  45. }}D. Salomon and G. Cormack. The disambiguation and scannerless parsing of complete character-level grammars for programming languages. Technical Report 95/06, Department of Computer Science, University of Manitoba, Winnipeg, Canada, 1995.Google ScholarGoogle Scholar
  46. }}D. J. Salomon and G. V. Cormack. Scannerless NSLR(1) parsing of programming languages. SIGPLAN Not., 24(7):170--178, 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. }}E. Scott and A. Johnstone. GLL parsing. In Workshop on Language Descriptions, Tools and Applications (LDTA'09), 2009.Google ScholarGoogle Scholar
  48. }}The Spoofax project. http://www.spoofax.org/.Google ScholarGoogle Scholar
  49. }}M. Tomita. Efficient Parsing for Natural Language: A Fast Algorithm for Practical Systems, volume 14. Kluwer Academic Publishers, 1988. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. }}E. Visser. A case study in optimizing parsing schemata by disambiguation filters. In IWPT, pages 210--224, 1997.Google ScholarGoogle Scholar
  51. }}E. Visser. Scannerless generalized-LR parsing. Technical Report P9707, Programming Research Group, University of Amsterdam, July 1997.Google ScholarGoogle Scholar
  52. }}E. Visser. Syntax Definition for Language Prototyping. PhD thesis, University of Amsterdam, September 1997.Google ScholarGoogle Scholar
  53. }}E. Visser. Meta-programming with concrete object syntax. In D. S. Batory, C. Consel, and W. Taha, editors, Generative Programming and Component Engineering, ACM SIGPLAN/SIGSOFT Conference, GPCE 2002, Pittsburgh, PA, USA, October 6-8, 2002, Proceedings, volume 2487 of Lecture Notes in Computer Science, pages 299--315. Springer, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. }}D. Younger. Recognition and parsing of context-free languages in time n3. Information and control, 10(2):189--208, 1967.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Pure and declarative syntax definition: paradise lost and regained

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!