Abstract
Contract systems, especially of the higher-order flavor, go hand in hand with blame. The pragmatic purpose of blame is to narrow down the code that a programmer needs to examine to locate the bug when the contract system discovers a contract violation. Or so the literature on higher-order contracts claims.
In reality, however, there is neither empirical nor theoretical evidence that connects blame with the location of bugs. The reputation of blame as a tool for weeding out bugs rests on anecdotes about how programmers use contracts to shift blame and their attention from one part of a program to another until they discover the source of the problem.
This paper aims to fill the apparent gap and shed light to the relation between blame and bugs. To that end, we introduce an empirical methodology for investigating whether, for a given contract system, it is possible to translate blame information to the location of bugs in a systematic manner. Our methodology is inspired by how programmers attempt to increase the precision of the contracts of a blamed component in order to shift blame to another component, which becomes the next candidate for containing the bug. In particular, we construct a framework that enables us to ask for a contract system whether (i) the process of blame shifting causes blame to eventually settle to the component that contains the bug; and (ii) every shift moves blame ``closer'' to the faulty component.
Our methodology offers a rigorous means for evaluating the pragmatics of contract systems, and we employ it to analyze Racket's contract system. Along the way, we uncover subtle points about the pragmatic meaning of contracts and blame in Racket: (i) the expressiveness of Racket's off-the-shelf contract language is not sufficient to narrow down the blamed portion of the code to the faulty component in all cases; and (ii) contracts that trigger state changes (even unexpectedly, perhaps in the runtime system's data structures or caches) interfere with program evaluation in subtle ways and thus blame shifting can lead programmers on a detour when searching for a bug. These points highlight how evaluations such as ours suggest fixes to language design.
Supplemental Material
- Umut A. Acar, Amal Ahmed, James Cheney, and Roly Perera. 2013. A core calculus for provenance. Journal of Computer Security 21, 6 (2013), 919–969.Google Scholar
Digital Library
- Hiralal Agrawal. 1991. Towards automatic debugging of computer programs. PhD thesis, Purdue University, SERC (1991).Google Scholar
- Hiralal Agrawal, Joseph R Horgan, Saul London, and W Eric Wong. 1995. Fault localization using execution slices and dataflow tests. In Proceedings of Sixth International Symposium on Software Reliability Engineering. IEEE, New York, NY, 143–151.Google Scholar
Cross Ref
- Amal Ahmed, Robert Bruce Findler, Jacob Matthews, and Philip Wadler. 2009. Blame for All. In Proceedings for the 1st Workshop on Script to Program Evolution. ACM, New York, NY, USA, 1–13.Google Scholar
Digital Library
- Matthias Blume and David McAllester. 2006. Sound and complete models of contracts. Journal of Functional Programming 16, 4-5 (2006), 375–414.Google Scholar
Digital Library
- James Cheney. 2011. A formal framework for provenance security. In Computer Security Foundations Symposium (CSF). IEEE, New York, NY, 281–293.Google Scholar
Digital Library
- James Cheney, Amal Ahmed, and Umut A. Acar. 2007. Provenance as dependency analysis. In International Symposium on Database Programming Languages. Springer, New York, NY, 138–152.Google Scholar
- Olaf Chitil, Dan McNeill, and Colin Runciman. 2003. Lazy assertions. In Symposium on Implementation and Application of Functional Languages. Springer, New York, NY, 1–19.Google Scholar
- Henry Coles. [n. d.]. Mutators Overview. http://pitest.org/quickstart/mutators/ . Accessed: 2019-07-02.Google Scholar
- Markus Degen, Peter Thiemann, and Stefan Wehr. 2008. Contract Monitoring and Call-by-name Evaluation. In Nordic Workshop on Programming Theory. Tallinn, Estonia.Google Scholar
- Markus Degen, Peter Thiemann, and Stefan Wehr. 2009. True lies: Lazy contracts for lazy languages (faithfulness is better than laziness). In 4. Arbeitstagung Programmiersprachen. Lübeck, Germany.Google Scholar
- Markus Degen, Peter Thiemann, and Stefan Wehr. 2010. Eager and delayed contract monitoring for call-by-value and call-by-name evaluation. The Journal of Logic and Algebraic Programming 79, 7 (2010), 515–549.Google Scholar
Cross Ref
- Markus Degen, Peter Thiemann, and Stefan Wehr. 2012. The Interaction of Contracts and Laziness. In Proceedings of the ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation. ACM, New York, NY, USA, 97–106.Google Scholar
Digital Library
- Richard A. DeMillo, Dana S. Guindi, Kim King, Mike M. McCracken, and Jefferson A. Offutt. 1988. An extended overview of the Mothra software testing environment. In Proceedings of the Second Workshop on Software Testing, Verification, and Analysis. IEEE, New York, NY, 142–151.Google Scholar
- Richard A. DeMillo, Richard J. Lipton, and Frederick G. Sayward. 1978. Hints on test data selection: Help for the practicing programmer. Computer 11, 4 (1978), 34–41.Google Scholar
Digital Library
- Christos Dimoulas and Matthias Felleisen. 2011. On Contract Satisfaction in a Higher-Order World. ACM Transactions on Programming Languages and Systems 33, 5 (2011), 16:1 – 16:29.Google Scholar
Digital Library
- Christos Dimoulas, Robert Bruce Findler, and Matthias Felleisen. 2013. Option contracts. In ACM Conference on ObjectOriented Programming Systems, Languages & Applications. ACM, New York, NY, 475–494.Google Scholar
Digital Library
- Christos Dimoulas, Robert Bruce Findler, Cormac Flanagan, and Matthias Felleisen. 2011. Correct Blame for Contracts: No More Scapegoating. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 215 – 226.Google Scholar
Digital Library
- Christos Dimoulas, Max S New, Robert Bruce Findler, and Matthias Felleisen. 2016. Oh Lord, please don’t let contracts be misunderstood (functional pearl). In ACM International Conference on Functional Programming. ACM, New York, NY, 117–131.Google Scholar
Digital Library
- Christos Dimoulas, Sam Tobin-Hochstadt, and Matthias Felleisen. 2012. Complete monitors for behavioral contracts. In European Symposium on Programming. Springer, New York, NY, 214–233.Google Scholar
Digital Library
- Tim Disney, Cormac Flanagan, and Jay McCarthy. 2011. Temporal higher-order contracts. In ACM International Conference on Functional Programming. ACM, New York, NY, 176–188.Google Scholar
Digital Library
- Daniel Feltey, Ben Greenman, Christophe Scholliers, Robert Bruce Findler, and Vincent St-Amour. 2018. Collapsible contracts: fixing a pathology of gradual typing. Proceedings of the ACM on Programming Languages 2, OOPSLA (2018), 133.Google Scholar
Digital Library
- Robert Bruce Findler and Matthias Blume. 2006. Contracts as Pairs of Projections. In Proceedings of the 8th International Symposium on Functional and Logic Programming. Springer, New York, NY, 226–241.Google Scholar
Digital Library
- Robert Bruce Findler and Matthias Felleisen. 2002. Contracts for Higher-Order Functions. In ACM International Conference on Functional Programming. ACM, New York, NY, 48–59.Google Scholar
- Robert Bruce Findler, Matthias Felleisen, and Matthias Blume. 2004. An investigation of contracts as projections. Technical Report TR-2004-02. University of Chicago, Computer Science Department.Google Scholar
- Robert Bruce Findler, Shu-yu Guo, and Anne Rogers. 2007. Lazy Contract Checking for Immutable Data Structures. In Revised Papers of the 16th International Workshop on Implementation of Functional Languages (IFL). Springer, New York, NY, 111–128.Google Scholar
- Ronald Garcia. 2013. Calculating Threesomes, with Blame. In ACM International Conference on Functional Programming. ACM, New York, NY, 417–428.Google Scholar
Digital Library
- Rahul Gopinath, Carlos Jensen, and Alex Groce. 2014. Mutations: How close are they to real faults?. In Software reliability engineering (ISSRE), 2014 IEEE 25th international symposium on. IEEE, New York, NY, 189–200.Google Scholar
- Michael Greenberg. 2015. Space-Efficient Manifest Contracts. In ACM Symposium on Principles of Programming Languages (POPL ’15). ACM, New York, NY, USA, 181–194.Google Scholar
- Michael Greenberg, Benjamin C. Pierce, and Stephanie Weirich. 2010. Contracts Made Manifest. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 353–364.Google Scholar
- Ben Greenman, Asumu Takikawa, Max S. New, Daniel Feltey, Robert Bruce Findler, Jan Vitek, and Matthias Felleisen. 2019. How to evaluate the performance of gradual type systems. Journal of Functional Programming 29 (2019), e4.Google Scholar
Cross Ref
- Bastiaan J Heeren. 2005. Top quality type error messages. Utrecht University.Google Scholar
- Phillip Heidegger, Annette Bieniusa, and Peter Thiemann. 2012. Access permission contracts for scripting languages. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 111–122.Google Scholar
Digital Library
- Ralf Hinze, Johan Jeuring, and Andres Löh. 2006. Typed contracts for functional programming. In International Symposium on Functional and Logic Programming. Springer, New York, NY, 208–225.Google Scholar
Digital Library
- Atsushi Igarashi, Peter Thiemann, Vasco T. Vasconcelos, and Philip Wadler. 2017. Gradual Session Types. Proceedings of the ACM on Programming Languages 1, ICFP, Article 38 (Aug. 2017), 28 pages.Google Scholar
Digital Library
- Limin Jia, Hannah Gommerstadt, and Frank Pfenning. 2016. Monitors and Blame Assignment for Higher-order Session Types. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 582–594.Google Scholar
- Yue Jia and Mark Harman. 2011. An analysis and survey of the development of mutation testing. IEEE transactions on software engineering 37, 5 (2011), 649–678.Google Scholar
Digital Library
- James A Jones, Mary Jean Harrold, and John Stasko. 2002. Visualization of test information to assist fault localization. In International Conference on Software Engineering. IEEE, New York, NY, 467–477.Google Scholar
Cross Ref
- René Just, Darioush Jalali, Laura Inozemtseva, Michael D Ernst, Reid Holmes, and Gordon Fraser. 2014. Are mutants a valid substitute for real faults in software testing?. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, New York, NY, 654–665.Google Scholar
Digital Library
- Matthias Keil and Peter Thiemann. 2015. Blame Assignment for Higher-order Contracts with Intersection and Union. In ACM International Conference on Functional Programming. ACM, New York, NY, USA, 375–386.Google Scholar
- Duc Le, Mohammad Amin Alipour, Rahul Gopinath, and Alex Groce. 2014. Mucheck: An extensible tool for mutation testing of Haskell programs. In Proceedings of the 2014 international symposium on software testing and analysis. ACM, New York, NY, 429–432.Google Scholar
Digital Library
- Richard J Lipton. 1971. Fault diagnosis of computer programs.Google Scholar
- Bertrand Meyer. 1988. Object-oriented Software Construction. Prentice Hall, London, UK.Google Scholar
Digital Library
- Bertrand Meyer. 1991. Design by Contract. In Advances in Object-Oriented Software Engineering. Prentice Hall, London, UK, 1–50.Google Scholar
- Bertrand Meyer. 1992. Eiffel: The Language. Prentice-Hall, Inc., Upper Saddle River, NJ, USA.Google Scholar
Digital Library
- Scott Moore, Christos Dimoulas, Robert Bruce Findler, Matthew Flatt, and Stephen Chong. 2016. Extensible access control with authorization contracts. In ACM Conference on Object-Oriented Programming Systems, Languages & Applications. ACM, New York, NY, 214–233.Google Scholar
Digital Library
- Scott Moore, Christos Dimoulas, Dan King, and Stephen Chong. 2014. SHILL: A Secure Shell Scripting Language. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation. USENIX Association, Berkeley, CA, USA, 183–199.Google Scholar
- Roly Perera, Umut A. Acar, James Cheney, and Paul Blain Levy. 2012. Functional programs that explain their work. In ACM International Conference on Functional Programming. ACM, New York, NY, 365–376.Google Scholar
Digital Library
- Christophe Scholliers, Éric Tanter, and Wolfgang De Meuter. 2015. Computational Contracts. Science of Computer Programming 98, P3 (Feb. 2015), 360–375.Google Scholar
Digital Library
- Ehud Y. Shapiro. 1983. Algorithmic Program DeBugging. MIT Press, Cambridge, MA, USA.Google Scholar
Digital Library
- Avraham Ever Shinnar. 2011. Safe and effective contracts. Harvard University, Boston, MA.Google Scholar
- Jeremy Siek, Ronald Garcia, and Walid Taha. 2009. Exploring the design space of higher-order casts. In European Symposium on Programming. Springer, New York, NY, 17–31.Google Scholar
Digital Library
- Jeremy Siek, Peter Thiemann, and Philip Wadler. 2015a. Blame and Coercion: Together Again for the First Time. In ACM Conference on Programming Language Design and Implementation. ACM, New York, NY, USA, 425–435.Google Scholar
- Jeremy G. Siek, Michael M. Vitousek, Matteo Cimini, Sam Tobin-Hochstadt, and Ronald Garcia. 2015b. Monotonic references for efficient gradual typing. In European Symposium on Programming Languages and Systems. Springer, New York, NY, 432–456.Google Scholar
Digital Library
- Jeremy G. Siek and Philip Wadler. 2010. Threesomes, with and Without Blame. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 365–376.Google Scholar
- T. Stephen Strickland and Matthias Felleisen. 2009a. Contracts for first-class modules. In Proceedings of the Symposium on Dynamic Languages. ACM, New York, NY, 27–38.Google Scholar
Digital Library
- T. Stephen Strickland and Matthias Felleisen. 2009b. Nested and Dynamic Contract Boundaries. In International Workshop on Implementation of Functional Languages (IFL). Springer, New York, NY, 141 – 158.Google Scholar
- T. Stephen Strickland, Sam Tobin-Hochstadt, Robert Bruce Findler, and Matthew Flatt. 2012. Chaperones and Impersonators: Run-time Support for Reasonable Interposition. In ACM Conference on Object-Oriented Programming Systems, Languages & Applications. ACM, New York, NY, 943–962.Google Scholar
Digital Library
- Cameron Swords, Amr Sabry, and Sam Tobin-Hochstadt. 2015. Expressing Contract Monitors As Patterns of Communication. In ACM International Conference on Functional Programming. ACM, New York, NY, 387–399.Google Scholar
- Asumu Takikawa, Daniel Feltey, Ben Greenman, Max S New, Jan Vitek, and Matthias Felleisen. 2016. Is sound gradual typing dead?. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 456–468.Google Scholar
Digital Library
- Asumu Takikawa, T Stephen Strickland, Christos Dimoulas, Sam Tobin-Hochstadt, and Matthias Felleisen. 2012. Gradual typing for first-class classes. In ACM Conference on Object-Oriented Programming Systems, Languages & Applications. ACM, New York, NY, 793–810.Google Scholar
Digital Library
- Ferdian Thung, David Lo, Lingxiao Jiang, et al. 2012. Are faults localizable?. In Proceedings of the 9th IEEE Working Conference on Mining Software Repositories. IEEE, New York, NY, 74–77.Google Scholar
- Michael M. Vitousek, Andrew M. Kent, Jeremy G. Siek, and Jim Baker. 2014. Design and Evaluation of Gradual Typing for Python. In Proceedings of the Symposium on Dynamic Languages. ACM, New York, NY, USA, 45–56.Google Scholar
Digital Library
- Michael M. Vitousek, Cameron Swords, and Jeremy G. Siek. 2017. Big types in little runtime: open-world soundness and collaborative blame for gradual type systems. In ACM Symposium on Principles of Programming Languages. ACM, New York, NY, 762–774.Google Scholar
- Philip Wadler and Robert Bruce Findler. 2009. Well-Typed Programs Can’t Be Blamed. In Proceedings of the 18th European Symposium on Programming Languages and Systems. Springer, New York, NY, 1–16.Google Scholar
Digital Library
- Lucas Waye, Stephen Chong, and Christos Dimoulas. 2017. Whip: higher-order contracts for modern services. Proceedings of the ACM on Programming Languages 1, ICFP (2017), 36.Google Scholar
Digital Library
- Jack Williams, J. Garrett Morris, and Philip Wadler. 2018. The Root Cause of Blame: Contracts for Intersection and Union Types. Proceedings of the ACM on Programming Languages 2, OOPSLA, Article 134 (Oct. 2018), 29 pages.Google Scholar
Digital Library
Index Terms
Does blame shifting work?
Recommendations
Blame assignment for higher-order contracts with intersection and union
ICFP '15We present an untyped calculus of blame assignment for a higher-order contract system with two new operators: intersection and union. The specification of these operators is based on the corresponding type theoretic constructions. This connection makes ...
Blame assignment for higher-order contracts with intersection and union
ICFP 2015: Proceedings of the 20th ACM SIGPLAN International Conference on Functional ProgrammingWe present an untyped calculus of blame assignment for a higher-order contract system with two new operators: intersection and union. The specification of these operators is based on the corresponding type theoretic constructions. This connection makes ...
An Explorative Comparison of Blame Attributions to Companion Robots Across Various Moral Dilemmas
HAI '18: Proceedings of the 6th International Conference on Human-Agent InteractionWe report results from an exploratory study with a humanoid robot asking participants (n = 30) to attribute blameworthiness to other robots that made decisions in moral dilemmas. Drawing from current research in machine ethics, we identify three ethical ...






Comments