Abstract
Integrated Development Environments (IDEs) increase programmer productivity, providing rapid, interactive feedback based on the syntax and semantics of a language. Unlike conventional parsing algorithms, scannerless generalized-LR parsing supports the full set of context-free grammars, which is closed under composition, and hence can parse languages composed from separate grammar modules. To apply this algorithm in an interactive environment, this article introduces a novel error recovery mechanism. Our approach is language independent, and relies on automatic derivation of recovery rules from grammars. By taking layout information into consideration it can efficiently suggest natural recovery suggestions.
- Barnard, D. T. and Holt, R. C. 1982. Hierarchic syntax error repair for lr grammars. Int. J. Comput. Inf. Sci. 11, 4, 231--258.Google Scholar
Cross Ref
- Bravenboer, M. and Visser, E. 2004. Concrete syntax for objects: Domain-Specific language embedding and assimilation without restrictions. In Proceedings of the 19th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA’04). J. M. Vlissides and D. C. Schmidt Eds., ACM Press, New York, 365--383. Google Scholar
Digital Library
- Bravenboer, M., Tanter, E., and Visser, E. 2006. Declarative, formal, and extensible syntax definition for AspectJ. In Proceedings of the 21th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA’06). P. L. Tarr and W. R. Cook Eds., ACM Press, New York, 209--228. Google Scholar
Digital Library
- Bravenboer, M., Dolstra, E., and Visser, E. 2007. Preventing injection attacks with syntax embeddings. In Proceedings of the 6th International Conference on Generative Programming and Component Engineering (GPCE’07). C. Consel and J. L. Lawall Eds., ACM Press, New York, 3--12. Google Scholar
Digital Library
- Bravenboer, M., Kalleberg, K. T., Vermaas, R., and Visser, E. 2008. Stratego/XT 0.17. A language and toolset for program transformation. Sci. Comput. Program. 72, 1--2, 52--70. Google Scholar
Digital Library
- Bravenboer, M., Dolstra, E., and Visser, E. 2010. Preventing injection attacks with syntax embeddings. Sci. Comput. Program. 75, 7, 473--495. Google Scholar
Digital Library
- Burke, M. G. and Fisher, G. A. 1987. A practical method for lr and ll syntactic error diagnosis and recovery. ACM Trans. Program. Lang. Syst. 9, 2, 164--197. Google Scholar
Digital Library
- Cerecke, C. 2002. Repairing syntax errors in lr-based parsers. In Proceedings of the Australian Computer Science Conference (ACSC’02). M. J. Oudshoorn Ed., CRPIT Series, vol. 4. Australian Computer Society, 17--22. Google Scholar
Digital Library
- Charles, P. 1991. A practical method for constructing efficient lalr(k) parsers with automatic error recovery. Ph.D. thesis, New York University. Google Scholar
Digital Library
- Charles, P., Fuhrer, R. M., and Sutton Jr., S. M. 2007. IMP: A meta-tooling platform for creating language-specific ides in Eclipse. In Proceedings of the Automated Software Engineering (ASE’07). R. E. K. Stirewalt, A. Egyed, and B. Fischer Eds., ACM Press, New York, 485--488. Google Scholar
Digital Library
- Corchuelo, R., Pérez, J. A., Cortés, A. R., and Toro, M. 2002. Repairing syntax errors in lr parsers. ACM Trans. Program. Lang. Syst. 24, 6, 698--710. Google Scholar
Digital Library
- de Jonge, M. and Visser, E. 2012. Automated evaluation of syntax error recovery. Tech. rep. TUD-SERG-2012-035, Delft University of Technology, Software Engineering Research Group, Delft, The Netherlands.Google Scholar
- de Jonge, M., Nilsson-Nyman, E., Kats, L. C. L., and Visser, E. 2009. Natural and flexible error recovery for generated parsers. In Proceedings of the 2nd International Conference on Software Language Engineering (SLE’09). M. van den Brand, D. Gasevic, and J. Gray Eds., Lecture Notes in Computer Science Series, vol. 5969, Springer, 204--223. Google Scholar
Digital Library
- Degano, P. and Priami, C. 1995. Comparison of syntactic error handling in lr parsers. Softw. Pract. Exper. 25, 6, 657--679. Google Scholar
Digital Library
- Ducasse, S., Nierstrasz, O., Schärli, N., Wuyts, R., and Black, A. 2006. Traits: A mechanism for fine-grained reuse. ACM Trans. Program. Lang. Syst. 28, 2, 331--388. Google Scholar
Digital Library
- Efftinge, S. and Voelter, M. 2006. oAW xText: A framework for textual dsls. In Proceedings of the Workshop on Modeling Symposium at Eclipse Summit.Google Scholar
- Fischer, C. N., Milton, D. R., and Quiring, S. B. 1980. Efficient ll(1) error correction and recovery using only insertions. Acta Inf. 13, 141--154.Google Scholar
Digital Library
- Ford, B. 2002. Packrat parsing: Simple, powerful, lazy, linear time. SIGPLAN Not. 37, 36--47. Google Scholar
Digital Library
- Fowler, M. 2005a. Language workbenches: The killer-app for domain specific languages? http://martinfowler.com/articles/languageWorkbench.html.Google Scholar
- Fowler, M. 2005b. PostIntelliJ. http://martinfowler.com/bliki/PostIntelliJ.html.Google Scholar
- Grimm, R. 2006. Better extensibility through modular syntax. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’06). 38--51. Google Scholar
Digital Library
- Grönniger, H., Krahn, H., Rumpe, B., Schindler, M., and Völkel, S. 2008. Monticore: A framework for the development of textual domain specific languages. In Proceedings of the International Conference on Software Engineering (ICSE’08). 925--926. Google Scholar
Digital Library
- Heering, J., Hendriks, P. R. H., Klint, P., and Rekers, J. 1989. The syntax definition formalism sdf. SIGPLAN Not. 24, 11, 43--75. Google Scholar
Digital Library
- Heidenreich, F., Johannes, J., Karol, S., Seifert, M., and Wende, C. 2009. Derivation and refinement of textual syntax for models. In Proceedings of the European Conference on Model-Driven Architecture-Foundations and Applications (ECMDA-FA’09). 114--129. Google Scholar
Digital Library
- Johnstone, A., Scott, E., and Economopoulos, G. 2004. Generalised parsing: Some costs. In Lecture Notes in Computer Science, vol. 2985, 89--103.Google Scholar
Cross Ref
- Jouault, F., Bézivin, J., and Kurtev, I. 2006. TCS: A dsl for the specification of textual concrete syntaxes in model engineering. In Proceedings of the Generative and Component Engineering (GPCE’06). ACM Press, New York, 249--254. Google Scholar
Digital Library
- Kats, L. C. L. and Visser, E. 2010. The Spoofax language workbench: Rules for declarative specification of languages and ides. In Proceedings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA’10). W. R. Cook, S. Clarke, and M. C. Rinard Eds., ACM Press, New York, 444--463. Google Scholar
Digital Library
- Kats, L. C. L., Bravenboer, M., and Visser, E. 2008. Mixing source and bytecode: A case for compilation by normalization. In Proceedings of the 23rd Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA’08). G. E. Harris Ed., ACM Press, New York, 91--108. Google Scholar
Digital Library
- Kats, L. C. L., de Jonge, M., Nilsson-Nyman, E., and Visser, E. 2009a. Providing rapid feedback in generated modular language environments. Adding error recovery to scannerless generalized-LR parsing. In Proceedings of the 24th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA’09). G. T. Leavens Ed., ACM Press, New York, 445--464. Google Scholar
Digital Library
- Kats, L. C. L., Sloane, A. M., and Visser, E. 2009b. Decorated attribute grammars: Attribute evaluation meets strategic programming. In Proceedings of the 18th International Conference on Compiler Construction (CC’09) held as Part of the Joint European Conferences on Theory and Practice of Software (ETAPS’09). O. de Moor and M. I. Schwartzbach Eds., Lecture Notes in Computer Science Series, vol. 5501, Springer, 142--157. Google Scholar
Digital Library
- Kats, L. C. L., Kalleberg, K. T., and Visser, E. 2010a. Domain-specific languages for composable editor plugins. In Proceedings of the 9th Workshop on Language Descriptions, Tools and Applications (LDTA’09). ENTCS Series, vol. 253, Elsevier. Google Scholar
Digital Library
- Kats, L. C. L., Visser, E., and Wachsmuth, G. 2010b. Pure and declarative syntax definition: Paradise lost and regained. In Proceedings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA’10). W. R. Cook, S. Clarke, and M. C. Rinard Eds., ACM Press, New York, 918--932. Google Scholar
Digital Library
- Kats, L. C. L., de Jonge, M., Nilsson-Nyman, E., and Visser, E. 2011. The permissive grammars project. http://strategoxt.org/Stratego/PermissiveGrammars.Google Scholar
- Kiczales, G., Lamping, J., Menhdhekar, A., Maeda, C., Lopes, C., Loingtier, J.-M., and Irwin, J. 1997. Aspect-oriented programming. In Proceedings of the European Conference on Object-Oriented Programming (ECOOP’97). M. Akşit and S. Matsuoka Eds., Lecture Notes in Computer Science, vol. 1241, Springer, 220--242.Google Scholar
- Klusener, S. and Lämmel, R. 2003. Deriving tolerant grammars from a base-line grammar. In International Conference on Software Maintenance (ICSM’03). IEEE Computer Society, 179--189. Google Scholar
Digital Library
- Krahn, H., Rumpe, B., and Völkel, S. 2007. Efficient editor generation for compositional dsls in Eclipse. In Proceedings of the 7th OOPSLA Workshop on Domain-Specific Modeling. Tech. rep. TR-38, University of Jyväskylä, 218--228.Google Scholar
- Krahn, H., Rumpe, B., and Völkel, S. 2008. MontiCore: Modular development of textual domain specific languages. In Proceedings of the International Conference on Objects, Model, Components and Patterns Europe (TOOLS EUROPE’08). R. Paige and B. Meyer Eds., Lecture Notes in Business Information Processing, vol. 11, Springer-Verlag, 297--315.Google Scholar
- Kuhn, T. and Thomann, O. 2006. Eclipse corner: Abstract syntax tree. http://eclipse.org/articles /article.php?file=Article-JavaCodeManipulation_AST/index.html.Google Scholar
- Lavie, A. and Tomita, M. 1993. GLR*-An efficient noise skipping parsing algorithm for context free grammars. In Proceedings of the 3rd International Workshop on Parsing Technologies. 123--134.Google Scholar
- Lévy, J.-P. 1971. Automatic correction of syntax errors in programming languages. Ph.D. thesis, Ithaca, NY.Google Scholar
- Mauney, J. and Fischer, C. 1988. Determining the extent of lookahead in syntactic error repair. ACM Trans. Program. Lang. Syst. 10, 3, 456--469. Google Scholar
Digital Library
- McKenzie, B. J., Yeatman, C., and Vere, L. D. 1995. Error repair in shift-reduce parsers. ACM Trans. Program. Lang. Syst. 17, 4, 672--689. Google Scholar
Digital Library
- Moonen, L. 2001. Generating robust parsers using island grammars. In Proceedings of the Working Conference on Reverse Engineering (WCRE’01). IEEE, 13--22. Google Scholar
Digital Library
- Moonen, L. 2002. Lightweight impact analysis using island grammars. In Proceedings of the 10th IEEE International Workshop of Program Comprehension. IEEE Computer Society, 219--228. Google Scholar
Digital Library
- Nilsson-Nyman, E., Ekman, T., and Hedin, G. 2009. Practical scope recovery using bridge parsing. In Proceedings of the International Conference on Software Language Engineering (SLE’08). D. Gasevic, R. Lämmel, and E. V. Wyk Eds., Lecture Notes in Computer Science, vol. 5452, Springer, 95--113. Google Scholar
Digital Library
- Pai, A. and Kieburtz, R. 1980. Global context recovery: A new strategy for syntactic error recovery by table-drive parsers. ACM Trans. Program. Lang. Syst. 2, 1, 18--41. Google Scholar
Digital Library
- Parr, T. and Fisher, K. 2011. Ll(*): The foundation of the antlr parser generator. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’11). 425--436. Google Scholar
Digital Library
- Parr, T. and Quong, R. 1995. ANTLR: A predicated-ll(k) parser generator. Softw. Pract. Exper. 25, 7, 789--810. Google Scholar
Digital Library
- Pennello, T. J. and DeRemer, F. 1978. A forward move algorithm for lr error recovery. In Proceedings of the ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’78). ACM Press, New York, 241--254. Google Scholar
Digital Library
- Rekers, J. and Koorn, W. 1991. Substring parsing for arbitrary context-free grammars. SIGPLAN Not. 26, 5, 59--66. Google Scholar
Digital Library
- Salomon, D. and Cormack, G. 1995. The disambiguation and scannerless parsing of complete character-level grammars for programming languages. Tech. rep. TR 95/06, Department of Computer Science, University of Manitoba, Winnipeg, Canada.Google Scholar
- Salomon, D. J. and Cormack, G. V. 1989. Scannerless nslr(1) parsing of programming languages. SIGPLAN Not. 24, 7, 170--178. Google Scholar
Digital Library
- Saunders, S., Fields, D. K., and Belayev, E. 2006. IntelliJ IDEA in Action. Manning.Google Scholar
- Schmitz, S. 2006. Modular syntax demands verification. Tech. rep. I3S/RR-2006-32-FR, Laboratoire I3S, Université de Nice-Sophia Antipolis, France.Google Scholar
- Schwerdfeger, A. C. and Van Wyk, E. R. 2009. Verifiable composition of deterministic grammars. SIGPLAN Not. 44, 6, 199--210. Google Scholar
Digital Library
- Swierstra, S. D. and Duponcheel, L. 1996. Deterministic, error-correcting combinator parsers. In Proceedings of the 2nd International School of Advanced Functional Programming. J. Launchbury et al., Eds., Lecture Notes in Computer Science, vol. 1129, Springer, 184--207. Google Scholar
Digital Library
- Synytskyy, N., Cordy, J., and Dean, T. 2003. Robust multilingual parsing using island grammars. In Proceedings of the Conference of the Centre for Advanced Studies on Collaborative Research. IBM Press, 266--278. Google Scholar
Digital Library
- Tai, K.-C. 1978. Syntactic error correction in programming languages. IEEE Trans. Softw. Engin. 4, 5, 414--425. Google Scholar
Digital Library
- Tomita, M. 1988. Efficient Parsing for Natural Language: A Fast Algorithm for Practical Systems. Vol. 14, Kluwer Academic Publishers.Google Scholar
- Valkering, R. 2007. Syntax error handling in scannerless generalized lr parsers. M.S. thesis, University of Amsterdam.Google Scholar
- van den Brand, M. G. J., de Jong, H., Klint, P., and Olivier, P. 2000. Efficient annotated terms. Softw. Pract. Exper. 30, 3, 259--291. Google Scholar
Digital Library
- van den Brand, M., Scheerder, J., Vinju, J. J., and Visser, E. 2002a. Disambiguation filters for scannerless generalized lr parsers. In Proceedings of the 11th International Conference on Compiler Construction (CC’02) held as Part of the Joint European Conferences on Theory and Practice of Software (ETAPS’02). R. N. Horspool Ed., Lecture Notes in Computer Science, vol. 2304, Springer, 143--158. Google Scholar
Digital Library
- van den Brand, M. G. J., Heering, J., Klint, P., and Olivier, P. A. 2002b. Compiling language definitions: The asf+sdf compiler. ACM Trans. Program. Lang. Syst. 24, 4, 334--368. Google Scholar
Digital Library
- van den Brand, M. G. J., Bruntink, M., Economopoulos, G. R., de Jong, H. A., Klint, P., Kooiker, T., van der Storm, T., and Vinju, J. J. 2007. Using the meta-environment for maintenance and renovation. In Proceedings of the European Conference on Software Maintenance and Reengineering (CSMR’07). IEEE Computer Society, 331--332. Google Scholar
Digital Library
- van Deursen, A. and Kuipers, T. 1999. Building documentation generators. In Proceedings of the IEEE International Conference on Software Maintenance (ICSM’99). IEEE Computer Society, 40. Google Scholar
Digital Library
- Visser, E. 1997a. A case study in optimizing parsing schemata by disambiguation filters. In Proceedings of the International Workshop on Parsing Technology (IWPT’97). 210--224.Google Scholar
- Visser, E. 1997b. Scannerless generalized-lr parsing. Tech. rep. P9707, Programming Research Group, University of Amsterdam.Google Scholar
- Visser, E. 1997c. Syntax definition for language prototyping. Ph.D. thesis, University of Amsterdam.Google Scholar
- Visser, E. 2002. Meta-Programming with concrete object syntax. In Proceedings of the ACM SIGPLAN-SIGSOFT Conference on Generative Programming and Component Engineering (GPCE’02). D. S. Batory, C. Consel, and W. Taha Eds., Lecture Notes in Computer Science, vol. 2487, Springer, 299--315. Google Scholar
Digital Library
- Waddington, D. and Yao, B. 2007. High-Fidelity c/c++ code transformation. Sci. Comput. Program. 68, 2, 64--78. Google Scholar
Digital Library
Index Terms
Natural and Flexible Error Recovery for Generated Modular Language Environments
Recommendations
Providing rapid feedback in generated modular language environments: adding error recovery to scannerless generalized-LR parsing
OOPSLA '09Integrated development environments (IDEs) increase programmer productivity, providing rapid, interactive feedback based on the syntax and semantics of a language. A heavy burden lies on developers of new languages to provide adequate IDE support. Code ...
Providing rapid feedback in generated modular language environments: adding error recovery to scannerless generalized-LR parsing
OOPSLA '09: Proceedings of the 24th ACM SIGPLAN conference on Object oriented programming systems languages and applicationsIntegrated development environments (IDEs) increase programmer productivity, providing rapid, interactive feedback based on the syntax and semantics of a language. A heavy burden lies on developers of new languages to provide adequate IDE support. Code ...
Towards automatic error recovery in parsing expression grammars
SBLP '18: Proceedings of the XXII Brazilian Symposium on Programming LanguagesError recovery is an essential feature for a parser that should be plugged in Integrated Development Environments (IDEs), which must build Abstract Syntax Trees (ASTs) even for syntactically invalid programs in order to offer features such as automated ...








Comments