skip to main content
research-article
Open Access
Artifacts Available
Artifacts Evaluated & Functional

Does blame shifting work?

Published:20 December 2019Publication History
Skip Abstract Section

Abstract

Contract systems, especially of the higher-order flavor, go hand in hand with blame. The pragmatic purpose of blame is to narrow down the code that a programmer needs to examine to locate the bug when the contract system discovers a contract violation. Or so the literature on higher-order contracts claims.

In reality, however, there is neither empirical nor theoretical evidence that connects blame with the location of bugs. The reputation of blame as a tool for weeding out bugs rests on anecdotes about how programmers use contracts to shift blame and their attention from one part of a program to another until they discover the source of the problem.

This paper aims to fill the apparent gap and shed light to the relation between blame and bugs. To that end, we introduce an empirical methodology for investigating whether, for a given contract system, it is possible to translate blame information to the location of bugs in a systematic manner. Our methodology is inspired by how programmers attempt to increase the precision of the contracts of a blamed component in order to shift blame to another component, which becomes the next candidate for containing the bug. In particular, we construct a framework that enables us to ask for a contract system whether (i) the process of blame shifting causes blame to eventually settle to the component that contains the bug; and (ii) every shift moves blame ``closer'' to the faulty component.

Our methodology offers a rigorous means for evaluating the pragmatics of contract systems, and we employ it to analyze Racket's contract system. Along the way, we uncover subtle points about the pragmatic meaning of contracts and blame in Racket: (i) the expressiveness of Racket's off-the-shelf contract language is not sufficient to narrow down the blamed portion of the code to the faulty component in all cases; and (ii) contracts that trigger state changes (even unexpectedly, perhaps in the runtime system's data structures or caches) interfere with program evaluation in subtle ways and thus blame shifting can lead programmers on a detour when searching for a bug. These points highlight how evaluations such as ours suggest fixes to language design.

Skip Supplemental Material Section

Supplemental Material

a65-lazarek.webm

References

  1. Umut A. Acar, Amal Ahmed, James Cheney, and Roly Perera. 2013. A core calculus for provenance. Journal of Computer Security 21, 6 (2013), 919–969.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Hiralal Agrawal. 1991. Towards automatic debugging of computer programs. PhD thesis, Purdue University, SERC (1991).Google ScholarGoogle Scholar
  3. Hiralal Agrawal, Joseph R Horgan, Saul London, and W Eric Wong. 1995. Fault localization using execution slices and dataflow tests. In Proceedings of Sixth International Symposium on Software Reliability Engineering. IEEE, New York, NY, 143–151.Google ScholarGoogle ScholarCross RefCross Ref
  4. Amal Ahmed, Robert Bruce Findler, Jacob Matthews, and Philip Wadler. 2009. Blame for All. In Proceedings for the 1st Workshop on Script to Program Evolution. ACM, New York, NY, USA, 1–13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Matthias Blume and David McAllester. 2006. Sound and complete models of contracts. Journal of Functional Programming 16, 4-5 (2006), 375–414.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. James Cheney. 2011. A formal framework for provenance security. In Computer Security Foundations Symposium (CSF). IEEE, New York, NY, 281–293.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. James Cheney, Amal Ahmed, and Umut A. Acar. 2007. Provenance as dependency analysis. In International Symposium on Database Programming Languages. Springer, New York, NY, 138–152.Google ScholarGoogle Scholar
  8. Olaf Chitil, Dan McNeill, and Colin Runciman. 2003. Lazy assertions. In Symposium on Implementation and Application of Functional Languages. Springer, New York, NY, 1–19.Google ScholarGoogle Scholar
  9. Henry Coles. [n. d.]. Mutators Overview. http://pitest.org/quickstart/mutators/ . Accessed: 2019-07-02.Google ScholarGoogle Scholar
  10. Markus Degen, Peter Thiemann, and Stefan Wehr. 2008. Contract Monitoring and Call-by-name Evaluation. In Nordic Workshop on Programming Theory. Tallinn, Estonia.Google ScholarGoogle Scholar
  11. Markus Degen, Peter Thiemann, and Stefan Wehr. 2009. True lies: Lazy contracts for lazy languages (faithfulness is better than laziness). In 4. Arbeitstagung Programmiersprachen. Lübeck, Germany.Google ScholarGoogle Scholar
  12. Markus Degen, Peter Thiemann, and Stefan Wehr. 2010. Eager and delayed contract monitoring for call-by-value and call-by-name evaluation. The Journal of Logic and Algebraic Programming 79, 7 (2010), 515–549.Google ScholarGoogle ScholarCross RefCross Ref
  13. Markus Degen, Peter Thiemann, and Stefan Wehr. 2012. The Interaction of Contracts and Laziness. In Proceedings of the ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation. ACM, New York, NY, USA, 97–106.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Richard A. DeMillo, Dana S. Guindi, Kim King, Mike M. McCracken, and Jefferson A. Offutt. 1988. An extended overview of the Mothra software testing environment. In Proceedings of the Second Workshop on Software Testing, Verification, and Analysis. IEEE, New York, NY, 142–151.Google ScholarGoogle Scholar
  15. Richard A. DeMillo, Richard J. Lipton, and Frederick G. Sayward. 1978. Hints on test data selection: Help for the practicing programmer. Computer 11, 4 (1978), 34–41.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Christos Dimoulas and Matthias Felleisen. 2011. On Contract Satisfaction in a Higher-Order World. ACM Transactions on Programming Languages and Systems 33, 5 (2011), 16:1 – 16:29.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Christos Dimoulas, Robert Bruce Findler, and Matthias Felleisen. 2013. Option contracts. In ACM Conference on ObjectOriented Programming Systems, Languages & Applications. ACM, New York, NY, 475–494.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Christos Dimoulas, Robert Bruce Findler, Cormac Flanagan, and Matthias Felleisen. 2011. Correct Blame for Contracts: No More Scapegoating. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 215 – 226.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Christos Dimoulas, Max S New, Robert Bruce Findler, and Matthias Felleisen. 2016. Oh Lord, please don’t let contracts be misunderstood (functional pearl). In ACM International Conference on Functional Programming. ACM, New York, NY, 117–131.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Christos Dimoulas, Sam Tobin-Hochstadt, and Matthias Felleisen. 2012. Complete monitors for behavioral contracts. In European Symposium on Programming. Springer, New York, NY, 214–233.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Tim Disney, Cormac Flanagan, and Jay McCarthy. 2011. Temporal higher-order contracts. In ACM International Conference on Functional Programming. ACM, New York, NY, 176–188.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Daniel Feltey, Ben Greenman, Christophe Scholliers, Robert Bruce Findler, and Vincent St-Amour. 2018. Collapsible contracts: fixing a pathology of gradual typing. Proceedings of the ACM on Programming Languages 2, OOPSLA (2018), 133.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Robert Bruce Findler and Matthias Blume. 2006. Contracts as Pairs of Projections. In Proceedings of the 8th International Symposium on Functional and Logic Programming. Springer, New York, NY, 226–241.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Robert Bruce Findler and Matthias Felleisen. 2002. Contracts for Higher-Order Functions. In ACM International Conference on Functional Programming. ACM, New York, NY, 48–59.Google ScholarGoogle Scholar
  25. Robert Bruce Findler, Matthias Felleisen, and Matthias Blume. 2004. An investigation of contracts as projections. Technical Report TR-2004-02. University of Chicago, Computer Science Department.Google ScholarGoogle Scholar
  26. Robert Bruce Findler, Shu-yu Guo, and Anne Rogers. 2007. Lazy Contract Checking for Immutable Data Structures. In Revised Papers of the 16th International Workshop on Implementation of Functional Languages (IFL). Springer, New York, NY, 111–128.Google ScholarGoogle Scholar
  27. Ronald Garcia. 2013. Calculating Threesomes, with Blame. In ACM International Conference on Functional Programming. ACM, New York, NY, 417–428.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Rahul Gopinath, Carlos Jensen, and Alex Groce. 2014. Mutations: How close are they to real faults?. In Software reliability engineering (ISSRE), 2014 IEEE 25th international symposium on. IEEE, New York, NY, 189–200.Google ScholarGoogle Scholar
  29. Michael Greenberg. 2015. Space-Efficient Manifest Contracts. In ACM Symposium on Principles of Programming Languages (POPL ’15). ACM, New York, NY, USA, 181–194.Google ScholarGoogle Scholar
  30. Michael Greenberg, Benjamin C. Pierce, and Stephanie Weirich. 2010. Contracts Made Manifest. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 353–364.Google ScholarGoogle Scholar
  31. Ben Greenman, Asumu Takikawa, Max S. New, Daniel Feltey, Robert Bruce Findler, Jan Vitek, and Matthias Felleisen. 2019. How to evaluate the performance of gradual type systems. Journal of Functional Programming 29 (2019), e4.Google ScholarGoogle ScholarCross RefCross Ref
  32. Bastiaan J Heeren. 2005. Top quality type error messages. Utrecht University.Google ScholarGoogle Scholar
  33. Phillip Heidegger, Annette Bieniusa, and Peter Thiemann. 2012. Access permission contracts for scripting languages. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 111–122.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Ralf Hinze, Johan Jeuring, and Andres Löh. 2006. Typed contracts for functional programming. In International Symposium on Functional and Logic Programming. Springer, New York, NY, 208–225.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Atsushi Igarashi, Peter Thiemann, Vasco T. Vasconcelos, and Philip Wadler. 2017. Gradual Session Types. Proceedings of the ACM on Programming Languages 1, ICFP, Article 38 (Aug. 2017), 28 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Limin Jia, Hannah Gommerstadt, and Frank Pfenning. 2016. Monitors and Blame Assignment for Higher-order Session Types. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 582–594.Google ScholarGoogle Scholar
  37. Yue Jia and Mark Harman. 2011. An analysis and survey of the development of mutation testing. IEEE transactions on software engineering 37, 5 (2011), 649–678.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. James A Jones, Mary Jean Harrold, and John Stasko. 2002. Visualization of test information to assist fault localization. In International Conference on Software Engineering. IEEE, New York, NY, 467–477.Google ScholarGoogle ScholarCross RefCross Ref
  39. René Just, Darioush Jalali, Laura Inozemtseva, Michael D Ernst, Reid Holmes, and Gordon Fraser. 2014. Are mutants a valid substitute for real faults in software testing?. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, New York, NY, 654–665.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Matthias Keil and Peter Thiemann. 2015. Blame Assignment for Higher-order Contracts with Intersection and Union. In ACM International Conference on Functional Programming. ACM, New York, NY, USA, 375–386.Google ScholarGoogle Scholar
  41. Duc Le, Mohammad Amin Alipour, Rahul Gopinath, and Alex Groce. 2014. Mucheck: An extensible tool for mutation testing of Haskell programs. In Proceedings of the 2014 international symposium on software testing and analysis. ACM, New York, NY, 429–432.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Richard J Lipton. 1971. Fault diagnosis of computer programs.Google ScholarGoogle Scholar
  43. Bertrand Meyer. 1988. Object-oriented Software Construction. Prentice Hall, London, UK.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Bertrand Meyer. 1991. Design by Contract. In Advances in Object-Oriented Software Engineering. Prentice Hall, London, UK, 1–50.Google ScholarGoogle Scholar
  45. Bertrand Meyer. 1992. Eiffel: The Language. Prentice-Hall, Inc., Upper Saddle River, NJ, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Scott Moore, Christos Dimoulas, Robert Bruce Findler, Matthew Flatt, and Stephen Chong. 2016. Extensible access control with authorization contracts. In ACM Conference on Object-Oriented Programming Systems, Languages & Applications. ACM, New York, NY, 214–233.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Scott Moore, Christos Dimoulas, Dan King, and Stephen Chong. 2014. SHILL: A Secure Shell Scripting Language. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation. USENIX Association, Berkeley, CA, USA, 183–199.Google ScholarGoogle Scholar
  48. Roly Perera, Umut A. Acar, James Cheney, and Paul Blain Levy. 2012. Functional programs that explain their work. In ACM International Conference on Functional Programming. ACM, New York, NY, 365–376.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Christophe Scholliers, Éric Tanter, and Wolfgang De Meuter. 2015. Computational Contracts. Science of Computer Programming 98, P3 (Feb. 2015), 360–375.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Ehud Y. Shapiro. 1983. Algorithmic Program DeBugging. MIT Press, Cambridge, MA, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Avraham Ever Shinnar. 2011. Safe and effective contracts. Harvard University, Boston, MA.Google ScholarGoogle Scholar
  52. Jeremy Siek, Ronald Garcia, and Walid Taha. 2009. Exploring the design space of higher-order casts. In European Symposium on Programming. Springer, New York, NY, 17–31.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Jeremy Siek, Peter Thiemann, and Philip Wadler. 2015a. Blame and Coercion: Together Again for the First Time. In ACM Conference on Programming Language Design and Implementation. ACM, New York, NY, USA, 425–435.Google ScholarGoogle Scholar
  54. Jeremy G. Siek, Michael M. Vitousek, Matteo Cimini, Sam Tobin-Hochstadt, and Ronald Garcia. 2015b. Monotonic references for efficient gradual typing. In European Symposium on Programming Languages and Systems. Springer, New York, NY, 432–456.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Jeremy G. Siek and Philip Wadler. 2010. Threesomes, with and Without Blame. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 365–376.Google ScholarGoogle Scholar
  56. T. Stephen Strickland and Matthias Felleisen. 2009a. Contracts for first-class modules. In Proceedings of the Symposium on Dynamic Languages. ACM, New York, NY, 27–38.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. T. Stephen Strickland and Matthias Felleisen. 2009b. Nested and Dynamic Contract Boundaries. In International Workshop on Implementation of Functional Languages (IFL). Springer, New York, NY, 141 – 158.Google ScholarGoogle Scholar
  58. T. Stephen Strickland, Sam Tobin-Hochstadt, Robert Bruce Findler, and Matthew Flatt. 2012. Chaperones and Impersonators: Run-time Support for Reasonable Interposition. In ACM Conference on Object-Oriented Programming Systems, Languages & Applications. ACM, New York, NY, 943–962.Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Cameron Swords, Amr Sabry, and Sam Tobin-Hochstadt. 2015. Expressing Contract Monitors As Patterns of Communication. In ACM International Conference on Functional Programming. ACM, New York, NY, 387–399.Google ScholarGoogle Scholar
  60. Asumu Takikawa, Daniel Feltey, Ben Greenman, Max S New, Jan Vitek, and Matthias Felleisen. 2016. Is sound gradual typing dead?. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 456–468.Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Asumu Takikawa, T Stephen Strickland, Christos Dimoulas, Sam Tobin-Hochstadt, and Matthias Felleisen. 2012. Gradual typing for first-class classes. In ACM Conference on Object-Oriented Programming Systems, Languages & Applications. ACM, New York, NY, 793–810.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Ferdian Thung, David Lo, Lingxiao Jiang, et al. 2012. Are faults localizable?. In Proceedings of the 9th IEEE Working Conference on Mining Software Repositories. IEEE, New York, NY, 74–77.Google ScholarGoogle Scholar
  63. Michael M. Vitousek, Andrew M. Kent, Jeremy G. Siek, and Jim Baker. 2014. Design and Evaluation of Gradual Typing for Python. In Proceedings of the Symposium on Dynamic Languages. ACM, New York, NY, USA, 45–56.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Michael M. Vitousek, Cameron Swords, and Jeremy G. Siek. 2017. Big types in little runtime: open-world soundness and collaborative blame for gradual type systems. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 762–774.Google ScholarGoogle Scholar
  65. Philip Wadler and Robert Bruce Findler. 2009. Well-Typed Programs Can’t Be Blamed. In Proceedings of the 18th European Symposium on Programming Languages and Systems. Springer, New York, NY, 1–16.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Lucas Waye, Stephen Chong, and Christos Dimoulas. 2017. Whip: higher-order contracts for modern services. Proceedings of the ACM on Programming Languages 1, ICFP (2017), 36.Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Jack Williams, J. Garrett Morris, and Philip Wadler. 2018. The Root Cause of Blame: Contracts for Intersection and Union Types. Proceedings of the ACM on Programming Languages 2, OOPSLA, Article 134 (Oct. 2018), 29 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Does blame shifting work?

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!