skip to main content
research-article
Open Access

How type errors were fixed and what students did?

Published:12 October 2017Publication History
Skip Abstract Section

Abstract

Providing better supports for debugging type errors has been an active research area in the last three decades. Numerous approaches from different perspectives have been developed. Most approaches work well under certain conditions only, for example, when type errors are caused by single leaves and when type annotations are correct. However, the research community is still unaware of which conditions hold in practice and what the real debugging situations look like. We address this problem with a study of 3 program data sets, which were written in different years, using different compilers, and were of diverse sizes. They include more than 55,000 programs, among which more than 2,700 are ill typed. We investigated all the ill-typed programs, and our results indicate that current error debugging support is far from sufficient in practice since only about 35% of all type errors were caused by single leaves. In addition, type annotations cannot always be trusted in error debuggers since about 30% of the time type errors were caused by wrong type annotations. Our study also provides many insights about the debugging behaviors of students in functional programming, which could be exploited for developing more effective error debuggers.

References

  1. Karen L. Bernstein and Eugene W. Stark. 1995. Debugging Type Errors. Technical Report. State University of New York at Stony Brook.Google ScholarGoogle Scholar
  2. Bernd Braßel. 2004. TypeHope: There is hope for your type errors. In Int. Workshop on Implementation of Functional Languages.Google ScholarGoogle Scholar
  3. Manuel MT Chakravarty and Gabriele Keller. 2004. The risks and benefits of teaching purely functional programming in first year. Journal of Functional Programming 14, 01 (2004), 113–123. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Christopher Chambers, Sheng Chen, Duc Le, and Christopher Scaffidi. 2012. The function, and dysfunction, of information sources in learning functional programming. Journal of Computing Sciences in Colleges 28, 1 (2012), 220–226.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. S. Chen and M. Erwig. 2014a. Counter-Factual Typing for Debugging Type Errors. In ACM SIGPLAN-SIGACT Symp. on Principles of Programming Languages. 583–594. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. S. Chen and M. Erwig. 2014b. Guided Type Debugging. In Int. Symp. on Functional and Logic Programming (LNCS 8475). 35–51. Google ScholarGoogle ScholarCross RefCross Ref
  7. Sheng Chen, Martin Erwig, and Karl Smeltzer. 2014. Let’s hear both sides: On combining type-error reporting tools. In 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 145–152. Google ScholarGoogle ScholarCross RefCross Ref
  8. S. Chen, M. Erwig, and E. Walkingshaw. 2012. An Error-Tolerant Type System for Variational Lambda Calculus. In ACM Int. Conf. on Functional Programming. 29–40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. S. Chen, M. Erwig, and E. Walkingshaw. 2014. Extending Type Inference to Variational Programs. ACM Trans. on Programming Languages and Systems 36, 1, Article 1 (2014), 54 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Olaf Chitil. 2001. Compositional Explanation of Types and Algorithmic Debugging of Type Errors. In ACM Int. Conf. on Functional Programming. 193–204. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Chris Clack and Colin Myers. 1995. The dys-functional student. In Funtional Programming Languages in Education. Springer, 289–309. Google ScholarGoogle ScholarCross RefCross Ref
  12. Paul Denny, Andrew Luxton-Reilly, and Ewan Tempero. 2012. All syntax errors are not equal. In Proceedings of the 17th ACM annual conference on Innovation and technology in computer science education. ACM, 75–80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Dominic Duggan and Frederick Bent. 1995. Explaining Type Inference. In Science of Computer Programming. 37–83.Google ScholarGoogle Scholar
  14. Hyunjun Eo, Oukseh Lee, and Kwangkeun Yi. 2004. Proofs of a set of hybrid let-polymorphic type inference algorithms. New Generation Computing 22, 1 (2004), 1–36. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. James B Fenwick Jr, Cindy Norris, Frank E Barry, Josh Rountree, Cole J Spicer, and Scott D Cheek. 2009. Another look at the behaviors of novice programmers. In ACM SIGCSE Bulletin, Vol. 41. ACM, 296–300.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Christian Haack and J. B. Wells. 2003. Type error slicing in implicitly typed higher-order languages. In European Symposium on Programming. 284–301. Google ScholarGoogle ScholarCross RefCross Ref
  17. J. Hage. Helium benchmark programs, (2002-2005). Private communication. (????).Google ScholarGoogle Scholar
  18. Jurriaan Hage and Bastiaan Heeren. 2007. Heuristics for Type Error Discovery and Recovery. In Implementation and Application of Functional Languages. 199–216. Google ScholarGoogle ScholarCross RefCross Ref
  19. Jurriaan Hage and Peter Van Keeken. 2006. Mining for Helium. Technical report UU-CS 2006-047 (2006).Google ScholarGoogle Scholar
  20. Jurriaan Hage and Peter van Keeken. 2009. Neon: A Library for Language Usage Analysis. In Software Language Engineering. Lecture Notes in Computer Science, Vol. 5452. 35–53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Bastiaan Heeren, Daan Leijen, and Arjan van IJzendoorn. 2003. Helium, for learning Haskell. In Proceedings of the 2003 ACM SIGPLAN workshop on Haskell (Haskell ’03). ACM, New York, NY, USA, 62–71. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Bastiaan J. Heeren. 2005. Top Quality Type Error Messages. Ph.D. Dissertation. Universiteit Utrecht, The Netherlands. http://www.cs.uu.nl/people/bastiaan/phdthesisGoogle ScholarGoogle Scholar
  23. Simon Peyton Jones. 2003. Haskell 98 language and libraries: the revised report. Cambridge University Press.Google ScholarGoogle Scholar
  24. Stef Joosten, Klaas Van Den Berg, and Gerrit Van Der Hoeven. 1993. Teaching functional programming to first-year students. Journal of Functional Programming 3, 01 (1993), 49–65. Google ScholarGoogle ScholarCross RefCross Ref
  25. Yang Jun, Greg Michaelson, and Phil Trinder. 2002. Explaining polymorphic types. Comput. J. 45, 4 (2002), 436–452. Google ScholarGoogle ScholarCross RefCross Ref
  26. Oukseh Lee and Kwangkeun Yi. 1998. Proofs about a folklore let-polymorphic type inference algorithm. ACM Trans. on Programming Languages and Systems 20, 4 (July 1998), 707–723. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Oukseh Lee and Kwangkeun Yi. 2000. A Generalized Let-Polymorphic Type Inference Algorithm. Technical Report. Technical Memorandum ROPAS-2000-5, Research on Program Analysis System, Korea Advanced Institute of Science and Technology.Google ScholarGoogle Scholar
  28. B. Lerner, M. Flower, Dan Grossman, and Craig Chambers. 2007. Searching for type-error messages. In ACM Int. Conf. on Programming Language Design and Implementation. 425–434. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Benjamin Lerner, Dan Grossman, and Craig Chambers. 2006. Seminal: searching for ML type-error messages. In Workshop on ML. 63–73. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Calvin Loncaric, Satish Chandra, Cole Schlesinger, and Manu Sridharan. 2016. A Practical Framework for Type Inference Error Explanation. In OOPSLA. 781–799. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Guillaume Marceau, Kathi Fisler, and Shriram Krishnamurthi. 2011a. Measuring the effectiveness of error messages designed for novice programmers. In Proceedings of the 42nd ACM technical symposium on Computer science education. ACM, 499–504. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Guillaume Marceau, Kathi Fisler, and Shriram Krishnamurthi. 2011b. Mind your language: on novices’ interactions with error messages. In Proceedings of the 10th SIGPLAN symposium on New ideas, new paradigms, and reflections on programming and software. ACM, 3–18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Bruce J McAdam. 2002. Repairing type errors in functional programs. Ph.D. Dissertation. University of Edinburgh. College of Science and Engineering. School of Informatics.Google ScholarGoogle Scholar
  34. Jonathan P Munson and Elizabeth A Schilling. 2016. Analyzing novice programmers’ response to compiler error messages. Journal of Computing Sciences in Colleges 31, 3 (2016), 53–61.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Robert Nieuwenhuis, Albert Oliveras, and Cesare Tinelli. 2006. Solving SAT and SAT Modulo Theories: From an Abstract Davis–Putnam–Logemann–Loveland Procedure to DPLL(T). J. ACM 53, 6 (Nov. 2006), 937–977. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Zvonimir Pavlinovic, Tim King, and Thomas Wies. 2014. Finding Minimum Type Error Sources. In OOPSLA. 525–542. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Zvonimir Pavlinovic, Tim King, and Thomas Wies. 2015. Practical SMT-based Type Error Localization. In ICFP. 412–423.Google ScholarGoogle Scholar
  38. Thomas Schilling. 2012. Constraint-Free type error slicing. In Trends in Functional Programming. Springer, 1–16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Jon Sneyers, Tom Schrijvers, and Bart Demoen. 2009. The Computational Power and Complexity of Constraint Handling Rules. ACM Trans. Program. Lang. Syst. 31, 2 (Feb. 2009), 8:1–8:42. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Peter J. Stuckey, Martin Sulzmann, and Jeremy Wazny. 2003. Interactive type debugging in Haskell. In ACM SIGPLAN Workshop on Haskell. 72–83. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Peter J. Stuckey, Martin Sulzmann, and Jeremy Wazny. 2004. Improving type error diagnosis. In ACM SIGPLAN Workshop on Haskell. 80–91. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Ville Tirronen. 2014. Study on difficulties and misconceptions with modern type systems. In Proceedings of the 2014 conference on Innovation & technology in computer science education. ACM, 303–308. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Ville Tirronen, SAMUEL UUSI-MÄKELÄ, and VILLE ISOMÖT TÖNEN. 2015. Understanding beginners’ mistakes with Haskell. Journal of Functional Programming 25 (2015), e11. Google ScholarGoogle ScholarCross RefCross Ref
  44. Kanae Tsushima and Kenichi Asai. 2013. An Embedded Type Debugger. Springer Berlin Heidelberg, Berlin, Heidelberg, 190–206. DOI: Google ScholarGoogle ScholarCross RefCross Ref
  45. Kanae Tsushima and Kenichi Asai. 2014. A Weighted Type-Error Slicer. Computer Software 31, 4 (2014), 131–148. DOI: Google ScholarGoogle ScholarCross RefCross Ref
  46. Kanae Tsushima and Olaf Chitil. 2014. Enumerating Counter-Factual Type Error Messages with an Existing Type Checker. In 16th Workshop on Programming and Programming Languages (PPL 2014).Google ScholarGoogle Scholar
  47. Peter van Keeken. 2006. Analyzing Helium Programs Obtained Through Logging–The process of mining novice Haskell programs. Master’s thesis. Department of Information and Computing Sciences, Utrecht University.Google ScholarGoogle Scholar
  48. Mitchell Wand. 1986. Finding the source of type errors. In ACM Symp. on Principles of Programming Languages. 38–43. DOI: Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Jeremy Richard Wazny. 2006. Type inference and type error diagnosis for Hindley/Milner with extensions. Ph.D. Dissertation. The University of Melbourne.Google ScholarGoogle Scholar
  50. Baijun Wu, John Peter Campora, and Sheng Chen. 2017. Learning User Friendly Type-error Messages. Proc. ACM Program. Lang. 1, OOPSLA, Article 106 (2017), 1 – 29.Google ScholarGoogle Scholar
  51. Jun Yang. 2000. Explaining Type Errors by Finding the Source of a Type Conflict. In Trends in Functional Programming. Intellect Books, 58–66.Google ScholarGoogle Scholar
  52. Jun Yang, Greg Michaelson, Phil Trinder, and JB Wells. 2000. Improved type error reporting. In In Proceedings of 12th International Workshop on Implementation of Functional Languages. Citeseer.Google ScholarGoogle Scholar
  53. Danfeng Zhang and Andrew C. Myers. 2014. Toward General Diagnosis of Static Errors. In ACM Symp. on Principles of Programming Languages. 569–581. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Danfeng Zhang, Andrew C. Myers, Dimitrios Vytiniotis, and Simon Peyton-Jones. 2015. Diagnosing Type Errors with Class. In ACM SIGPLAN Conference on Programming Language Design and Implementation. 12–21. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. How type errors were fixed and what students did?

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Proceedings of the ACM on Programming Languages
        Proceedings of the ACM on Programming Languages  Volume 1, Issue OOPSLA
        October 2017
        1786 pages
        EISSN:2475-1421
        DOI:10.1145/3152284
        Issue’s Table of Contents

        Copyright © 2017 Owner/Author

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 12 October 2017
        Published in pacmpl Volume 1, Issue OOPSLA

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!