Abstract
In this paper, we present a notion of observability and controllability in the context of software testing and debugging. Our view of observability is based on the ability of developers, testers, and debuggers to trace back a data dependency chain and observe the value of a variable by starting from a set of variables that are naturally observable (e.g., input/output variables). Likewise, our view of controllability enables one to modify and control the value of a variable through a data dependency chain by starting from a set of variables that can be modified (e.g., input variables). Consequently, the problem that we study in this paper is to identify the minimum number of variables that have to be made observable/controllable in order for a tester or debugger to observe/control the value of another set of variables of interest, given the source code. We show that our problem is an instance of the well-known abstract diagnosis problem, where the objective is to find the minimum number of faulty components in a digital circuit, given the system description and value of input/output variables. We show that our problem is NP-complete even if the length of data dependencies is at most 2. In order to cope with the inevitable exponential complexity, we propose a mapping from the general problem, where the length of data dependency chains is unknown a priori, to integer linear programming. Our method is fully implemented in a tool chain for MISRA-C compliant source codes. Our experiments with several real-world applications show that in average, a significant number of debugging points can be reduced using our methods. This result is our motivation to apply our approach in debugging and instrumentation of embedded software, where changes must be minimal as they can perturb the timing constraints and resource consumption. Another interesting application of our results is in data logging of non-terminating embedded systems, where axillary data storage devices are slow and have limited size.
- R. Abreu and A. J. C. van Gemund. A low-cost approximate minimal hitting set algorithm and its application to model-based diagnosis. In Abstraction, Reformulation, and Approximation (SARA), 2009.Google Scholar
- T. Ball amd and J. R Larus. Optimally profiling and tracing programs. ACM Transactions on Programming Languages and Systems (TOPLAS), 16(4):1319--1360, 1994. Google Scholar
Digital Library
- Robert V. Binder. Design for Testability in Object-Oriented Systems. Communications of the ACM, 37(9):87--101, 1994. Google Scholar
Digital Library
- David Binkley and Mark Harman. A Survey of Empirical Results on Program Slicing. In Advances in Computers, pages 105--178, 2003.Google Scholar
- Ron Cytron, Jeanne Ferrante, Barry K. Rosen, Mark N. Wegman, and F. Kenneth Zadeck. Efficiently Computing Static Single Assignment Form and the Control Dependence Graph. ACM Transactions on Programming Languages and Systems, 13(4):451--490, 1991. Google Scholar
Digital Library
- Huy Vu Do, Chantal Robach, and Michel Delaunay. Automatic Testability Analysis for Data-Flow Designs of Reactive Systems. In Proceedings of the First International Workshop on Testability Assessment, pages 52--61, 2004.Google Scholar
- A. Fijany and F. Vatan. New high-performance algorithmic solution for diagnosis problem. In IEEE Aerospace Conference (IEEEAC), 2005.Google Scholar
Cross Ref
- Roy S. Freedman. Testability of Software Components. IEEE Transactions on Software Engineering, 17(6):553--564, 1991. Google Scholar
Digital Library
- Hideo Fujiwara. Logic Testing and Design for Testability. Massachusetts Institute of Technology, Cambridge, MA, USA, 1985. Google Scholar
Digital Library
- Hideo Fujiwara. Computational Complexity of Controllability/Observability Problems for Combinational Circuits. IEEE Transactions on Computers, 39(6):762--767, 1990. Google Scholar
Digital Library
- M. R. Guthaus, J. S. Ringenberg, D. Ernst, T. M. Austin, T. Mudge, and R. B. Brown. MiBench: A free, commercially representative embedded benchmark suite. In IEEE International Workshop on In Workload Characterization (WWC), pages 3--14, 2001. Google Scholar
Digital Library
- R. M. Karp. Reducibility among combinatorial problems. In Symposium on Complexity of Computer Computations, pages 85--103, 1972.Google Scholar
Cross Ref
- Chris Lattner and Vikram Adve. LLVM: A Compilation Framework for Lifelong Program Analysis and Transformation. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback directed and runtime Optimization, page 75, 2004. Google Scholar
Digital Library
- Gavin McCall. Misra-C: 2004. MIRA Limited, Warwickshire, United Kingdom, 2004.Google Scholar
- Thanh Binh Nguyen, Michel Delaunay, and Chantal Robach. Testability Analysis Applied to Embedded Data-flow Software. In Proceedings of the Third International Conference on Quality Software, page 351, Washington, DC, USA, 2003. IEEE Computer Society. Google Scholar
Digital Library
- Thanh Binh Nguyen, Michel Delaunay, and Chantal Robach. Testability Analysis of Data-Flow Software. Electronic Notes in Theoretical Computer Science, 116:213--225, 2005.Google Scholar
Digital Library
- R. Reiter. A theory of diagnosis from first principles. Artificial Intelligence, 32(1):57--95, 1987. Google Scholar
Digital Library
- Werner Schutz. The Testability of Distributed Real-Time Systems. Kluwer Academic Publishers, Norwell, MA, USA, 1993. Google Scholar
Digital Library
- Werner Schutz. Fundamental Issues in Testing Distributed Real-Time Systems. Real-Time Systems, 7(2):129--157, 1994. Google Scholar
Digital Library
- Henrik Thane and Hans Hansson. Towards Systematic Testing of Distributed Real-Time Systems. In Proceedings of the 20th IEEE Real-Time Systems Symposium, pages 360--369, Washington, DC, USA, 1999. IEEE Computer Society. Google Scholar
Digital Library
- Henrik Thane, Daniel Sundmark, Joel Huselius, and Anders Pettersson. Replay Debugging of Real-Time Systems Using Time Machines. In Proceedings of the 17th International Symposium on Parallel and Distributed Processing, page 8, Washington, DC, USA, 2003. IEEE Computer Society. Google Scholar
Digital Library
- Frank Tip. A Survey of Program Slicing Techniques. Technical report, IBM, Amsterdam, The Netherlands, The Netherlands, 1994. Google Scholar
Digital Library
- Yves Le Traon, Farid Ouabdesselam, and Chantal Robach. Analyzing Testability on Data Flow Designs. In Proceedings of the 11th International Symposium on Software Reliability Engineering, page 162, Washington, DC, USA, 2000. IEEE Computer Society. Google Scholar
Digital Library
- Yves Le Traon and Chantal Robach. Testability Measurements for Data Flow Designs. In Proceedings of the 4th International Symposium on Software Metrics, page 91, Washington, DC, USA, 1997. IEEE Computer Society. Google Scholar
Digital Library
- Jeffrey M. Voas and Keith W. Miller. Semantic Metrics for Software Testability. Journal of Systems and Software, 20(3):207--216, 1993. Google Scholar
Digital Library
Index Terms
Software debugging and testing using the abstract diagnosis theory
Recommendations
Software debugging and testing using the abstract diagnosis theory
LCTES '11: Proceedings of the 2011 SIGPLAN/SIGBED conference on Languages, compilers and tools for embedded systemsIn this paper, we present a notion of observability and controllability in the context of software testing and debugging. Our view of observability is based on the ability of developers, testers, and debuggers to trace back a data dependency chain and ...
An empirical analysis of the distribution of unit test smells and their impact on software maintenance
ICSM '12: Proceedings of the 2012 IEEE International Conference on Software Maintenance (ICSM)Unit testing represents a key activity in software development and maintenance. Test suites with high internal quality facilitate maintenance activities, such as code comprehension and regression testing. Several guidelines have been proposed to help ...
Who’s debugging the debuggers? exposing debug information bugs in optimized binaries
ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating SystemsDespite the advancements in software testing, bugs still plague deployed software and result in crashes in production. When debugging issues —sometimes caused by “heisenbugs”— there is the need to interpret core dumps and reproduce the issue offline on ...







Comments