skip to main content
research-article

Software debugging and testing using the abstract diagnosis theory

Published:11 April 2011Publication History
Skip Abstract Section

Abstract

In this paper, we present a notion of observability and controllability in the context of software testing and debugging. Our view of observability is based on the ability of developers, testers, and debuggers to trace back a data dependency chain and observe the value of a variable by starting from a set of variables that are naturally observable (e.g., input/output variables). Likewise, our view of controllability enables one to modify and control the value of a variable through a data dependency chain by starting from a set of variables that can be modified (e.g., input variables). Consequently, the problem that we study in this paper is to identify the minimum number of variables that have to be made observable/controllable in order for a tester or debugger to observe/control the value of another set of variables of interest, given the source code. We show that our problem is an instance of the well-known abstract diagnosis problem, where the objective is to find the minimum number of faulty components in a digital circuit, given the system description and value of input/output variables. We show that our problem is NP-complete even if the length of data dependencies is at most 2. In order to cope with the inevitable exponential complexity, we propose a mapping from the general problem, where the length of data dependency chains is unknown a priori, to integer linear programming. Our method is fully implemented in a tool chain for MISRA-C compliant source codes. Our experiments with several real-world applications show that in average, a significant number of debugging points can be reduced using our methods. This result is our motivation to apply our approach in debugging and instrumentation of embedded software, where changes must be minimal as they can perturb the timing constraints and resource consumption. Another interesting application of our results is in data logging of non-terminating embedded systems, where axillary data storage devices are slow and have limited size.

References

  1. R. Abreu and A. J. C. van Gemund. A low-cost approximate minimal hitting set algorithm and its application to model-based diagnosis. In Abstraction, Reformulation, and Approximation (SARA), 2009.Google ScholarGoogle Scholar
  2. T. Ball amd and J. R Larus. Optimally profiling and tracing programs. ACM Transactions on Programming Languages and Systems (TOPLAS), 16(4):1319--1360, 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Robert V. Binder. Design for Testability in Object-Oriented Systems. Communications of the ACM, 37(9):87--101, 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. David Binkley and Mark Harman. A Survey of Empirical Results on Program Slicing. In Advances in Computers, pages 105--178, 2003.Google ScholarGoogle Scholar
  5. Ron Cytron, Jeanne Ferrante, Barry K. Rosen, Mark N. Wegman, and F. Kenneth Zadeck. Efficiently Computing Static Single Assignment Form and the Control Dependence Graph. ACM Transactions on Programming Languages and Systems, 13(4):451--490, 1991. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Huy Vu Do, Chantal Robach, and Michel Delaunay. Automatic Testability Analysis for Data-Flow Designs of Reactive Systems. In Proceedings of the First International Workshop on Testability Assessment, pages 52--61, 2004.Google ScholarGoogle Scholar
  7. A. Fijany and F. Vatan. New high-performance algorithmic solution for diagnosis problem. In IEEE Aerospace Conference (IEEEAC), 2005.Google ScholarGoogle ScholarCross RefCross Ref
  8. Roy S. Freedman. Testability of Software Components. IEEE Transactions on Software Engineering, 17(6):553--564, 1991. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Hideo Fujiwara. Logic Testing and Design for Testability. Massachusetts Institute of Technology, Cambridge, MA, USA, 1985. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Hideo Fujiwara. Computational Complexity of Controllability/Observability Problems for Combinational Circuits. IEEE Transactions on Computers, 39(6):762--767, 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. M. R. Guthaus, J. S. Ringenberg, D. Ernst, T. M. Austin, T. Mudge, and R. B. Brown. MiBench: A free, commercially representative embedded benchmark suite. In IEEE International Workshop on In Workload Characterization (WWC), pages 3--14, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. R. M. Karp. Reducibility among combinatorial problems. In Symposium on Complexity of Computer Computations, pages 85--103, 1972.Google ScholarGoogle ScholarCross RefCross Ref
  13. Chris Lattner and Vikram Adve. LLVM: A Compilation Framework for Lifelong Program Analysis and Transformation. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback directed and runtime Optimization, page 75, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Gavin McCall. Misra-C: 2004. MIRA Limited, Warwickshire, United Kingdom, 2004.Google ScholarGoogle Scholar
  15. Thanh Binh Nguyen, Michel Delaunay, and Chantal Robach. Testability Analysis Applied to Embedded Data-flow Software. In Proceedings of the Third International Conference on Quality Software, page 351, Washington, DC, USA, 2003. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Thanh Binh Nguyen, Michel Delaunay, and Chantal Robach. Testability Analysis of Data-Flow Software. Electronic Notes in Theoretical Computer Science, 116:213--225, 2005.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. R. Reiter. A theory of diagnosis from first principles. Artificial Intelligence, 32(1):57--95, 1987. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Werner Schutz. The Testability of Distributed Real-Time Systems. Kluwer Academic Publishers, Norwell, MA, USA, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Werner Schutz. Fundamental Issues in Testing Distributed Real-Time Systems. Real-Time Systems, 7(2):129--157, 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Henrik Thane and Hans Hansson. Towards Systematic Testing of Distributed Real-Time Systems. In Proceedings of the 20th IEEE Real-Time Systems Symposium, pages 360--369, Washington, DC, USA, 1999. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Henrik Thane, Daniel Sundmark, Joel Huselius, and Anders Pettersson. Replay Debugging of Real-Time Systems Using Time Machines. In Proceedings of the 17th International Symposium on Parallel and Distributed Processing, page 8, Washington, DC, USA, 2003. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Frank Tip. A Survey of Program Slicing Techniques. Technical report, IBM, Amsterdam, The Netherlands, The Netherlands, 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Yves Le Traon, Farid Ouabdesselam, and Chantal Robach. Analyzing Testability on Data Flow Designs. In Proceedings of the 11th International Symposium on Software Reliability Engineering, page 162, Washington, DC, USA, 2000. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Yves Le Traon and Chantal Robach. Testability Measurements for Data Flow Designs. In Proceedings of the 4th International Symposium on Software Metrics, page 91, Washington, DC, USA, 1997. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jeffrey M. Voas and Keith W. Miller. Semantic Metrics for Software Testability. Journal of Systems and Software, 20(3):207--216, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Software debugging and testing using the abstract diagnosis theory

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 46, Issue 5
        LCTES '10
        May 2011
        170 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/2016603
        Issue’s Table of Contents
        • cover image ACM Conferences
          LCTES '11: Proceedings of the 2011 SIGPLAN/SIGBED conference on Languages, compilers and tools for embedded systems
          April 2011
          182 pages
          ISBN:9781450305556
          DOI:10.1145/1967677

        Copyright © 2011 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 11 April 2011

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!