skip to main content
research-article
Open Access

Highly illogical, Kirk: spotting type mismatches in the large despite broken contracts, unsound types, and too many linters

Published:31 October 2022Publication History
Skip Abstract Section

Abstract

The DefinitelyTyped repository hosts type declarations for thousands of JavaScript libraries. Given the lack of formal connection between the types and the corresponding code, a natural question is are the types right? An equally important question, as DefinitelyTyped and the libraries it supports change over time, is how can we keep the types from becoming wrong? In this paper we offer Scotty, a tool that detects mismatches between the types and code in the Definitely-Typed repository. More specifically, Scotty checks each package by converting its types into contracts and installing the contracts on the boundary between the library and its test suite. Running the test suite in this environment can reveal mismatches between the types and the JavaScript code. As automation and generality are both essential if such a tool is going to remain useful in the long term, we focus on techniques that sacrifice completeness, instead preferring to avoid false positives. Scotty currently handles about 26% of the 8006 packages on DefinitelyTyped (61% of the packages whose code is available and whose test suite passes). Perhaps unsurprisingly, running the tests with these contracts in place revealed many errors in Definitely-Typed. More surprisingly, despite the inherent limitations of the techniques we use, this exercise led to one hundred accepted pull requests that fix errors in DefinitelyTyped, demonstrating the value of this approach for the long-term maintenance of DefinitelyTyped. It also revealed a number of lessons about working in the JavaScript ecosystem and how details beyond the semantics of the language can be surprisingly important. Best of all, it also revealed a few places where programmers preferred incorrect types, suggesting some avenues of research to improve TypeScript.

References

  1. Amal Ahmed, Robert Bruce Findler, Jacob Matthews, and Philip Wadler. Blame for all. In Proc. Workshop on Script to Program Evolution, 2009. https://doi.org/10.1145/1570506.1570507 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Amal Ahmed, Dustin Jamner, Jeremy Siek, and Philip Wadler. Theorems for Free for Free: Parametricity, With and Without Types. In Proc. ACM International Conference on Functional Programming, 2017. https://doi.org/10.1145/3110283 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Spenser Bauman, Carl Friedrich Bolz-Tereick, Jeremy Siek, and Sam Tobin-Hochstadt. Sound Gradual Typing: Only Mostly Dead. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, 2017. https://doi.org/10.1145/3133878 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Moritz Beller, Radjino Bholanath, Shane Mcintosh, and Andy Zaidman. Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software. In Proc. IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), pp. 470-481, 2016. https://doi.org/10.1109/SANER. 2016. 105 Google ScholarGoogle ScholarCross RefCross Ref
  5. Gavin Bierman, Martín Abadi, and Mads Torgersen. Understanding TypeScript. In Proc. European Conference on Object-Oriented Programming, 2014. https://doi.org/10.1007/978-3-662-44202-9_11 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Giuseppe Castagna and Victor Lanvin. Gradual Typing with Union and Intersection Types. In Proc. ACM International Conference on Functional Programming, 2017. https://doi.org/10.1145/3110285 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Avik Chaudhuri. Flow: Abstract Interpretation of JavaScript for Type Checking and Beyond (Invited Talk). In Proc. Workshop on Programming Languages and Analysis for Security, 2016. https://doi.org/10.1145/ 2993600.2996280 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Fernando Cristiani and Peter Thiemann. Generation of TypeScript Declaration Files from JavaScript Code. In Proc. International Conference on Managed Programming Languages and Runtimes, 2021. https://doi.org/ 10.1145/3475738.3480941 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ecma International. Standard ECMA-262-ECMAScript Language Specification. 2015. http://www.ecmainternational. org/ecma-262/6.0/ Google ScholarGoogle Scholar
  10. Asger Feldthaus and Anders Møller. Checking Correctness of TypeScript Interfaces for JavaScript Libraries. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, 2014. https://doi.org/10.1145/2660193.2660215 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Daniel Feltey, Ben Greenman, Christophe Scholliers, Robert Bruce Findler, and Vincent St-Amour. Collapsible Contracts: Fixing a Pathology of Gradual Typing. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, 2018. https://doi.org/10.1145/3276503 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Robert Bruce Findler and Matthias Felleisen. Contracts for Higher-Order Functions. In Proc. ACM International Conference on Functional Programming, 2002. https://doi.org/10.1145/2502508.2502521 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Frolin S. Ocariza Jr., Karthik Pattabiraman, and Benjamin Zorn. JavaScript Errors in the Wild: An Empirical Study. In Proc. IEEE 22nd International Symposium on Software Reliability Engineering, 2011. https: //doi.org/10.1109/ISSRE. 2011.28 Google ScholarGoogle ScholarCross RefCross Ref
  14. Zheng Gao, Christian Bird, and Earl T. Barr. To Type or Not to Type: Quantifying Detectable Bugs in JavaScript. In Proc. International Conference on Software Engineering, 2017. https://doi.org/10.1109/ICSE. 2017.75 Google ScholarGoogle ScholarCross RefCross Ref
  15. Ben Greenman, Asumu Takikawa, Max S. New, Daniel Feltey, Robert Bruce Findler, Jan Vitek, and Matthias Felleisen. How to evaluate the performance of gradual type systems. Journal of Functional Programming, 2019. https://doi.org/10.1017/S0956796818000217 Google ScholarGoogle ScholarCross RefCross Ref
  16. Jessica Gronski, Kenneth Knowles, Aaron Tomb, Stephen N. Freund, and Cormac Flanagan. Sage: Hybrid Checking for Flexible Specifications. In Proc. Workshop on Scheme and Functional Programming, pp. 93-104, 2006. http://www.schemeworkshop.org/ 2006 /06-freund.pdf Google ScholarGoogle Scholar
  17. Arjun Guha, Jacob Matthews, Robert Bruce Findler, and Shriram Krishnamurthi. Relationally-Parametric Polymorphic Contracts. In Proc. Dynamic Languages Symposium, 2007. https://doi.org/10.1145/1297081. 1297089 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Simon Holm Jensen, Anders Møller, and Peter Thiemann. Type Analysis for JavaScript. In Proc. International Conference on Static Analysis, 2009. https://doi.org/10.1007/978-3-642-03237-0_17 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Matthias Keil, Sankha Narayan Guria, Andreas Schlegel, Manuel Gefken, and Peter Thiemann. Transparent Object Proxies in JavaScript. In Proc. European Conference on Object-Oriented Programming, 2015. https: //doi.org/10.4230/LIPIcs.ECOOP. 2015.149 Google ScholarGoogle ScholarCross RefCross Ref
  20. Matthias Keil and Peter Thiemann. On the Proxy Identity Crisis. 2013. https://arxiv.org/abs/1312.5429 Google ScholarGoogle Scholar
  21. Matthias Keil and Peter Thiemann. Blame Assignment for Higher-Order Contracts with Intersection and Union. In Proc. ACM International Conference on Functional Programming, 2015. https://doi.org/10.1145/ 2858949.2784737 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Erik Krogh Kristensen and Anders Møller. Inference and Evolution of TypeScript Declaration Files. In Proc. International Conference on the Fundamental Approaches to Software Engineering, 2017a. https://doi.org/ 10.1007/978-3-662-54494-5_6 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Erik Krogh Kristensen and Anders Møller. Type Test Scripts for TypeScript Testing. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, 2017b. https://doi.org/10.1145/ 3133914 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Erik Krogh Kristensen and Anders Møller. Reasonably-Most-General Clients for JavaScript Library Analysis. In Proc. International Conference on Software Engineering, 2019. https://doi.org/10.1109/ICSE. 2019.00026 Google ScholarGoogle ScholarCross RefCross Ref
  25. Rabee Malik, Jibesh Patra, and Michael Pradel. NL2Type: Inferring JavaScript Function Types from Natural Language Information. In Proc. International Conference on Software Engineering, 2019. https://dl.acm. org/doi/10.1109/ICSE. 2019.00045 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Jacob Matthews and Amal Ahmed. Parametric polymorphism through run-time sealing, or, Theorems for low, low prices ! In Proc. European Symposium on Programming, 2008. https://doi.org/10.1007/978-3-540-78739-6_2 Google ScholarGoogle ScholarCross RefCross Ref
  27. Charalambos Mitropoulos. Employing Diferent Program Analysis Methods to Study Bug Evolution. In Proc. International Symposium on the Foundations of Software Engineering, 2019. https://doi.org/10.1145/3338906. 3342489 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Cameron Moy, PhC. Nguyundefinedn, Sam Tobin-Hochstadt, and David Van Horn. Corpse Reviver : Sound and Eficient Gradual Typing via Contract Verification. In Proc. ACM Symposium on Principles of Programming Languages, 2021. https://doi.org/10.1145/3434334 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Fabian Muehlboeck and Ross Tate. Sound Gradual Typing is Nominally Alive and Well. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, 2017. https://doi.org/10.1145/ 3133880 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Anders Møller, Benjamin Barslev Nielsen, and Martin Toldam Torp. Detecting Locations in JavaScript Programs Afected by Breaking Library Changes. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, 2020. https://doi.org/10.1145/3428255 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Benjamin Barslev Nielsen, Martin Toldam Torp, and Anders Møller. Semantic Patches for Adaptation of JavaScript Programs to Evolving Libraries. In Proc. International Conference on Software Engineering, 2021. https://doi.org/10.1109/ICSE43902. 2021.00020 Google ScholarGoogle ScholarCross RefCross Ref
  32. Aseem Rastogi, Nikhil Swamy, Cand Bierman, Gavin Fournet, and Panagiotis Vekris. Safe & Eficient Gradual Typing for TypeScript. In Proc. ACM Symposium on Principles of Programming Languages, 2015. https: //doi.org/10.1145/2676726.2676971 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. N. Rinetzky, A. Poetzsch-Hefter, G. Ramalingam, M. Sagiv, and E. Yahav. Modular Shape Analysis for Dynamically Encapsulated Programs. In Proc. European Symposium on Programming, 2007. https: //dl.acm.org/doi/10.5555/1762174.1762197 Google ScholarGoogle ScholarCross RefCross Ref
  34. Koushik Sen, Swaroop Kalasapur, Tasneem Brutch, and Simon Gibbs. Jalangi: A Selective Record-Replay and Dynamic Analysis Framework for JavaScript. In Proc. International Symposium on the Foundations of Software Engineering, 2013. https://doi.org/10.1145/2491411.2491447 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Jeremy G. Siek and Walid Taha. Gradual typing for functional languages. In Proc. Workshop on Scheme and Functional Programming, 2006. http://scheme2006.cs.uchicago.edu/13-siek.pdf Google ScholarGoogle Scholar
  36. Asumu Takikawa, Daniel Feltey, Ben Greenman, Max S. New, Jan Vitek, and Matthias Felleisen. Is Sound Gradual Typing Dead? In Proc. ACM Symposium on Principles of Programming Languages, 2016. https: //doi.org/10.1145/2837614.2837630 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. The TypeScript Handbook. 2019. https://www.typescriptlang.org/docs/handbook/type-compatibility. html# comparing-two-functions Google ScholarGoogle Scholar
  38. Sam Tobin-Hochstadt and Matthias Felleisen. Interlanguage Migration: From Scripts to Programs. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, 2006. https: //doi.org/10.1145/1176617.1176755 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Kristín Fjóla Tómasdóttir, Maurício Aniche, and Arie Van Deursen. Why and How JavaScript Developers Use Linters. In Proc. ACM/IEEE International Conference on Automated Software Engineering, pp. 578-589, 2017. https://dl.acm.org/doi/10.5555/3155562.3155634 Google ScholarGoogle Scholar
  40. Kristín Fjóla Tómasdóttir, Maurício Aniche, and Arie Van Deursen. The Adoption of JavaScript Linters in Practice: A Case Study on ESLint. IEEE Transactions on Software Engineering 46 ( 8 ), pp. 863-891, 2020. https://doi.org/10.1109/TSE. 2018.2871058 Google ScholarGoogle ScholarCross RefCross Ref
  41. Tom Van Cutsem and Mark Miller. Trustworthy Proxies-Virtualizing Objects with Invariants. In Proc. European Conference on Object-Oriented Programming, 2013. https://research.google/pubs/pub40736/ Google ScholarGoogle Scholar
  42. Michael M. Vitousek, Andrew M. Kent, Jeremy Siek, and Jin G. Baker. Design and Evaluation of Gradual Typing for Python. In Proc. Dynamic Languages Symposium, 2014. https://doi.org/10.1145/2661088.2661101 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Jack Williams, Garrett J. Morris, and Philip Wadler. The Root Cause of Blame: Contracts for Intersection and Union Types. ACM Transactions on Programming Languages and Systems, 2018. https://doi.org/10.1145/ 3276504 Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Jack Williams, J. Garrett Morris, Philip Wadler, and Jakub Zalewski. Mixed Messages: Measuring Conformance and Non-Interference in TypeScript. In Proc. European Conference on Object-Oriented Programming, 2017. https://www.pure.ed.ac.uk/ws/portalfiles/portal/38670500/ecoop17_mixed_messages_2.pdf Google ScholarGoogle Scholar

Index Terms

  1. Highly illogical, Kirk: spotting type mismatches in the large despite broken contracts, unsound types, and too many linters

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Article Metrics

      • Downloads (Last 12 months)104
      • Downloads (Last 6 weeks)14

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!