Abstract
Providing better supports for debugging type errors has been an active research area in the last three decades. Numerous approaches from different perspectives have been developed. Most approaches work well under certain conditions only, for example, when type errors are caused by single leaves and when type annotations are correct. However, the research community is still unaware of which conditions hold in practice and what the real debugging situations look like. We address this problem with a study of 3 program data sets, which were written in different years, using different compilers, and were of diverse sizes. They include more than 55,000 programs, among which more than 2,700 are ill typed. We investigated all the ill-typed programs, and our results indicate that current error debugging support is far from sufficient in practice since only about 35% of all type errors were caused by single leaves. In addition, type annotations cannot always be trusted in error debuggers since about 30% of the time type errors were caused by wrong type annotations. Our study also provides many insights about the debugging behaviors of students in functional programming, which could be exploited for developing more effective error debuggers.
- Karen L. Bernstein and Eugene W. Stark. 1995. Debugging Type Errors. Technical Report. State University of New York at Stony Brook.Google Scholar
- Bernd Braßel. 2004. TypeHope: There is hope for your type errors. In Int. Workshop on Implementation of Functional Languages.Google Scholar
- Manuel MT Chakravarty and Gabriele Keller. 2004. The risks and benefits of teaching purely functional programming in first year. Journal of Functional Programming 14, 01 (2004), 113–123. Google Scholar
Digital Library
- Christopher Chambers, Sheng Chen, Duc Le, and Christopher Scaffidi. 2012. The function, and dysfunction, of information sources in learning functional programming. Journal of Computing Sciences in Colleges 28, 1 (2012), 220–226.Google Scholar
Digital Library
- S. Chen and M. Erwig. 2014a. Counter-Factual Typing for Debugging Type Errors. In ACM SIGPLAN-SIGACT Symp. on Principles of Programming Languages. 583–594. Google Scholar
Digital Library
- S. Chen and M. Erwig. 2014b. Guided Type Debugging. In Int. Symp. on Functional and Logic Programming (LNCS 8475). 35–51. Google Scholar
Cross Ref
- Sheng Chen, Martin Erwig, and Karl Smeltzer. 2014. Let’s hear both sides: On combining type-error reporting tools. In 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 145–152. Google Scholar
Cross Ref
- S. Chen, M. Erwig, and E. Walkingshaw. 2012. An Error-Tolerant Type System for Variational Lambda Calculus. In ACM Int. Conf. on Functional Programming. 29–40. Google Scholar
Digital Library
- S. Chen, M. Erwig, and E. Walkingshaw. 2014. Extending Type Inference to Variational Programs. ACM Trans. on Programming Languages and Systems 36, 1, Article 1 (2014), 54 pages.Google Scholar
Digital Library
- Olaf Chitil. 2001. Compositional Explanation of Types and Algorithmic Debugging of Type Errors. In ACM Int. Conf. on Functional Programming. 193–204. Google Scholar
Digital Library
- Chris Clack and Colin Myers. 1995. The dys-functional student. In Funtional Programming Languages in Education. Springer, 289–309. Google Scholar
Cross Ref
- Paul Denny, Andrew Luxton-Reilly, and Ewan Tempero. 2012. All syntax errors are not equal. In Proceedings of the 17th ACM annual conference on Innovation and technology in computer science education. ACM, 75–80. Google Scholar
Digital Library
- Dominic Duggan and Frederick Bent. 1995. Explaining Type Inference. In Science of Computer Programming. 37–83.Google Scholar
- Hyunjun Eo, Oukseh Lee, and Kwangkeun Yi. 2004. Proofs of a set of hybrid let-polymorphic type inference algorithms. New Generation Computing 22, 1 (2004), 1–36. DOI: Google Scholar
Digital Library
- James B Fenwick Jr, Cindy Norris, Frank E Barry, Josh Rountree, Cole J Spicer, and Scott D Cheek. 2009. Another look at the behaviors of novice programmers. In ACM SIGCSE Bulletin, Vol. 41. ACM, 296–300.Google Scholar
Digital Library
- Christian Haack and J. B. Wells. 2003. Type error slicing in implicitly typed higher-order languages. In European Symposium on Programming. 284–301. Google Scholar
Cross Ref
- J. Hage. Helium benchmark programs, (2002-2005). Private communication. (????).Google Scholar
- Jurriaan Hage and Bastiaan Heeren. 2007. Heuristics for Type Error Discovery and Recovery. In Implementation and Application of Functional Languages. 199–216. Google Scholar
Cross Ref
- Jurriaan Hage and Peter Van Keeken. 2006. Mining for Helium. Technical report UU-CS 2006-047 (2006).Google Scholar
- Jurriaan Hage and Peter van Keeken. 2009. Neon: A Library for Language Usage Analysis. In Software Language Engineering. Lecture Notes in Computer Science, Vol. 5452. 35–53. Google Scholar
Digital Library
- Bastiaan Heeren, Daan Leijen, and Arjan van IJzendoorn. 2003. Helium, for learning Haskell. In Proceedings of the 2003 ACM SIGPLAN workshop on Haskell (Haskell ’03). ACM, New York, NY, USA, 62–71. DOI: Google Scholar
Digital Library
- Bastiaan J. Heeren. 2005. Top Quality Type Error Messages. Ph.D. Dissertation. Universiteit Utrecht, The Netherlands. http://www.cs.uu.nl/people/bastiaan/phdthesisGoogle Scholar
- Simon Peyton Jones. 2003. Haskell 98 language and libraries: the revised report. Cambridge University Press.Google Scholar
- Stef Joosten, Klaas Van Den Berg, and Gerrit Van Der Hoeven. 1993. Teaching functional programming to first-year students. Journal of Functional Programming 3, 01 (1993), 49–65. Google Scholar
Cross Ref
- Yang Jun, Greg Michaelson, and Phil Trinder. 2002. Explaining polymorphic types. Comput. J. 45, 4 (2002), 436–452. Google Scholar
Cross Ref
- Oukseh Lee and Kwangkeun Yi. 1998. Proofs about a folklore let-polymorphic type inference algorithm. ACM Trans. on Programming Languages and Systems 20, 4 (July 1998), 707–723. DOI: Google Scholar
Digital Library
- Oukseh Lee and Kwangkeun Yi. 2000. A Generalized Let-Polymorphic Type Inference Algorithm. Technical Report. Technical Memorandum ROPAS-2000-5, Research on Program Analysis System, Korea Advanced Institute of Science and Technology.Google Scholar
- B. Lerner, M. Flower, Dan Grossman, and Craig Chambers. 2007. Searching for type-error messages. In ACM Int. Conf. on Programming Language Design and Implementation. 425–434. DOI: Google Scholar
Digital Library
- Benjamin Lerner, Dan Grossman, and Craig Chambers. 2006. Seminal: searching for ML type-error messages. In Workshop on ML. 63–73. DOI: Google Scholar
Digital Library
- Calvin Loncaric, Satish Chandra, Cole Schlesinger, and Manu Sridharan. 2016. A Practical Framework for Type Inference Error Explanation. In OOPSLA. 781–799. Google Scholar
Digital Library
- Guillaume Marceau, Kathi Fisler, and Shriram Krishnamurthi. 2011a. Measuring the effectiveness of error messages designed for novice programmers. In Proceedings of the 42nd ACM technical symposium on Computer science education. ACM, 499–504. Google Scholar
Digital Library
- Guillaume Marceau, Kathi Fisler, and Shriram Krishnamurthi. 2011b. Mind your language: on novices’ interactions with error messages. In Proceedings of the 10th SIGPLAN symposium on New ideas, new paradigms, and reflections on programming and software. ACM, 3–18. Google Scholar
Digital Library
- Bruce J McAdam. 2002. Repairing type errors in functional programs. Ph.D. Dissertation. University of Edinburgh. College of Science and Engineering. School of Informatics.Google Scholar
- Jonathan P Munson and Elizabeth A Schilling. 2016. Analyzing novice programmers’ response to compiler error messages. Journal of Computing Sciences in Colleges 31, 3 (2016), 53–61.Google Scholar
Digital Library
- Robert Nieuwenhuis, Albert Oliveras, and Cesare Tinelli. 2006. Solving SAT and SAT Modulo Theories: From an Abstract Davis–Putnam–Logemann–Loveland Procedure to DPLL(T). J. ACM 53, 6 (Nov. 2006), 937–977. DOI: Google Scholar
Digital Library
- Zvonimir Pavlinovic, Tim King, and Thomas Wies. 2014. Finding Minimum Type Error Sources. In OOPSLA. 525–542. Google Scholar
Digital Library
- Zvonimir Pavlinovic, Tim King, and Thomas Wies. 2015. Practical SMT-based Type Error Localization. In ICFP. 412–423.Google Scholar
- Thomas Schilling. 2012. Constraint-Free type error slicing. In Trends in Functional Programming. Springer, 1–16. Google Scholar
Digital Library
- Jon Sneyers, Tom Schrijvers, and Bart Demoen. 2009. The Computational Power and Complexity of Constraint Handling Rules. ACM Trans. Program. Lang. Syst. 31, 2 (Feb. 2009), 8:1–8:42. DOI: Google Scholar
Digital Library
- Peter J. Stuckey, Martin Sulzmann, and Jeremy Wazny. 2003. Interactive type debugging in Haskell. In ACM SIGPLAN Workshop on Haskell. 72–83. DOI: Google Scholar
Digital Library
- Peter J. Stuckey, Martin Sulzmann, and Jeremy Wazny. 2004. Improving type error diagnosis. In ACM SIGPLAN Workshop on Haskell. 80–91. DOI: Google Scholar
Digital Library
- Ville Tirronen. 2014. Study on difficulties and misconceptions with modern type systems. In Proceedings of the 2014 conference on Innovation & technology in computer science education. ACM, 303–308. Google Scholar
Digital Library
- Ville Tirronen, SAMUEL UUSI-MÄKELÄ, and VILLE ISOMÖT TÖNEN. 2015. Understanding beginners’ mistakes with Haskell. Journal of Functional Programming 25 (2015), e11. Google Scholar
Cross Ref
- Kanae Tsushima and Kenichi Asai. 2013. An Embedded Type Debugger. Springer Berlin Heidelberg, Berlin, Heidelberg, 190–206. DOI: Google Scholar
Cross Ref
- Kanae Tsushima and Kenichi Asai. 2014. A Weighted Type-Error Slicer. Computer Software 31, 4 (2014), 131–148. DOI: Google Scholar
Cross Ref
- Kanae Tsushima and Olaf Chitil. 2014. Enumerating Counter-Factual Type Error Messages with an Existing Type Checker. In 16th Workshop on Programming and Programming Languages (PPL 2014).Google Scholar
- Peter van Keeken. 2006. Analyzing Helium Programs Obtained Through Logging–The process of mining novice Haskell programs. Master’s thesis. Department of Information and Computing Sciences, Utrecht University.Google Scholar
- Mitchell Wand. 1986. Finding the source of type errors. In ACM Symp. on Principles of Programming Languages. 38–43. DOI: Google Scholar
Digital Library
- Jeremy Richard Wazny. 2006. Type inference and type error diagnosis for Hindley/Milner with extensions. Ph.D. Dissertation. The University of Melbourne.Google Scholar
- Baijun Wu, John Peter Campora, and Sheng Chen. 2017. Learning User Friendly Type-error Messages. Proc. ACM Program. Lang. 1, OOPSLA, Article 106 (2017), 1 – 29.Google Scholar
- Jun Yang. 2000. Explaining Type Errors by Finding the Source of a Type Conflict. In Trends in Functional Programming. Intellect Books, 58–66.Google Scholar
- Jun Yang, Greg Michaelson, Phil Trinder, and JB Wells. 2000. Improved type error reporting. In In Proceedings of 12th International Workshop on Implementation of Functional Languages. Citeseer.Google Scholar
- Danfeng Zhang and Andrew C. Myers. 2014. Toward General Diagnosis of Static Errors. In ACM Symp. on Principles of Programming Languages. 569–581. Google Scholar
Digital Library
- Danfeng Zhang, Andrew C. Myers, Dimitrios Vytiniotis, and Simon Peyton-Jones. 2015. Diagnosing Type Errors with Class. In ACM SIGPLAN Conference on Programming Language Design and Implementation. 12–21. Google Scholar
Digital Library
Index Terms
How type errors were fixed and what students did?
Recommendations
Counter-factual typing for debugging type errors
POPL '14: Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming LanguagesChanging a program in response to a type error plays an important part in modern software development. However, the generation of good type error messages remains a problem for highly expressive type systems. Existing approaches often suffer from a lack ...
Counter-factual typing for debugging type errors
POPL '14Changing a program in response to a type error plays an important part in modern software development. However, the generation of good type error messages remains a problem for highly expressive type systems. Existing approaches often suffer from a lack ...
Discriminative sum types locate the source of type errors
We propose a type system for locating the source of type errors in an applied lambda calculus with ML-style polymorphism. The system is based on discriminative sum types---known from work on soft typing---with annotation subtyping and recursive types. ...






Comments