Abstract
Mutation testing is a program-transformation technique that injects artificial bugs to check whether the existing test suite can detect them. However, the costs of using mutation testing are usually high, hindering its use in industry. Useless mutants (equivalent and duplicated) contribute to increase costs. Previous research has focused mainly on detecting useless mutants only after they are generated and compiled. In this paper, we introduce a strategy to help developers with deriving rules to avoid the generation of useless mutants. To use our strategy, we pass as input a set of programs. For each program, we also need a passing test suite and a set of mutants. As output, our strategy yields a set of useless mutants candidates. After manually confirming that the mutants classified by our strategy as "useless" are indeed useless, we derive rules that can avoid their generation and thus decrease costs. To the best of our knowledge, we introduce 37 new rules that can avoid useless mutants right before their generation. We then implement a subset of these rules in the MUJAVA mutation testing tool. Since our rules have been derived based on artificial and small Java programs, we take our MUJAVA version embedded with our rules and execute it in industrial-scale projects. Our rules reduced the number of mutants by almost 13% on average. Our results are promising because (i) we avoid useless mutants generation; (ii) our strategy can help with identifying more rules in case we set it to use more complex Java programs; and (iii) our MUJAVA version has only a subset of the rules we derived.
- 2017. PITest Mutation Testing Tool for Java. (2017). http://pitest.org/ Accessed: 2017-05-20.Google Scholar
- Konstantinos Adamopoulos, Mark Harman, and Robert M Hierons. 2004. How to overcome the equivalent mutant problem and achieve tailored selective mutation using co-evolution. In Genetic and evolutionary computation conference. 1338-1349.Google Scholar
- Jr. Allen Troy Acree. 1980. On Mutation. Ph.D. Dissertation. Georgia Institute of Technology.Google Scholar
- Cyrille Artho and Lei Ma. 2016. Classification of randomly generated test cases. In Software Analysis, Evolution, and Reengineering (SANER), IEEE 23rd International Conference on. 29-32.Google Scholar
- Douglas Baldwin and Frederick Sayward. 1979. Heuristics for Determining Equivalence of Program Mutations. Technical Report. DTIC Document.Google Scholar
- Timothy Budd and Dana Angluin. 1982. Two notions of correctness and their relation to testing. Acta Informatica 18, 1 (1982), 31-45. Google Scholar
Digital Library
- Richard DeMillo, Richard Lipton, and Frederick Sayward. 1978. Hints on Test Data Selection: Help for the Practicing Programmer. Computer 11, 4 (1978), 34-41. Google Scholar
Digital Library
- Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering. 416-419. Google Scholar
Digital Library
- Bernhard JM Grün, David Schuler, and Andreas Zeller. 2009. The impact of equivalent mutants. In Software Testing, Verification and Validation Workshops. ICSTW. International Conference on. 192-199. Google Scholar
Digital Library
- Rob Hierons, Mark Harman, and Sebastian Danicic. 1999. Using program slicing to assist in the detection of equivalent mutants. Software Testing, Verification and Reliability 9, 4 (1999), 233-262.Google Scholar
Cross Ref
- Daniel Jackson. 2012. Software Abstractions: Logic, Language, and Analysis. 2nd edition. The MIT Press. Google Scholar
Digital Library
- Daniel Jackson, Ian Schechter, and Hya Shlyahter. 2000. Alcoa: the Alloy constraint analyzer. In Proceedings of the 31st International Conference on Software Engineering (ICSE). IEEE Computer Society, 730-733. Google Scholar
Digital Library
- Yue Jia and Mark Harman. 2011. An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37, 5 (2011), 649-678. Google Scholar
Digital Library
- René Just, Darioush Jalali, Laura Inozemtseva, Michael Ernst, Reid Holmes, and Gordon Fraser. 2014. Are mutants a valid substitute for real faults in software testing?. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 654-665. Google Scholar
Digital Library
- René Just, Bob Kurtz, and Paul Ammann. 2017. Inferring Mutant Utility from Program Context. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA). Google Scholar
Digital Library
- Rene Just, Franz Schweiggert, and Gregory Kapfhammer. 2011. MAJOR: An efficient and extensible tool for mutation analysis in a Java compiler. In Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering. 612-615. Google Scholar
Digital Library
- Marinos Kintis and Nicos Malevris. 2015. MEDIC: A static analysis framework for equivalent mutant identification. Information and Software Technology 68 (2015), 1-17.Google Scholar
Digital Library
- Marinos Kintis, Mike Papadakis, Yue Jia, Nicos Malevris, Yves Le Traon, and Mark Harman. 2017. Detecting Trivial Mutant Equivalences via Compiler Optimisations. IEEE Transactions on Software Engineering PP, 99 (2017), 1-1.Google Scholar
- Marinos Kintis, Mike Papadakis, Andreas Papadopoulos, Evangelos Valvis, and Nicos Malevris. 2016. Analysing and Comparing the Effectiveness of Mutation Testing Tools: A Manual Study. In Source Code Analysis and Manipulation (SCAM), 16th International Working Conference on. 147-156.Google Scholar
- Yu-Seung Ma, Jeff Offutt, and Yong Rae Kwon. 2005. MuJava: An automated class mutation system. Software Testing, Verification and Reliability 15, 2 (2005), 97-133. Google Scholar
Cross Ref
- Lech Madeyski, Wojciech Orzeszyna, Richard Torkar, and Mariusz Jozala. 2014. Overcoming the Equivalent Mutant Problem: A Systematic Literature Review and a Comparative Experiment of Second Order Mutation. IEEE Transactions on Software Engineering 40, 1 (2014), 23-42. Google Scholar
Digital Library
- Melina Mongiovi. 2017. Scaling Testing of Refactoring Engines. Ph.D. Dissertation. Federal University of Campina Grande (UFCG).Google Scholar
- Melina Mongiovi, Gustavo Mendes, Rohit Gheyi, Gustavo Soares, and Márcio Ribeiro. 2014. Scaling testing of refactoring engines. In Software Maintenance and Evolution (ICSME), IEEE International Conference on. 371-380. Google Scholar
Digital Library
- Jeff Offutt, Yu-Seung Ma, and Yong-Rae Kwon. 2006. The Class-level Mutants of MuJava. In Proceedings of the 2006 International Workshop on Automation of Software Test. 78-84. Google Scholar
Digital Library
- Jeff Offutt and Jie Pan. 1996. Detecting equivalent mutants and the feasible path problem. In Computer Assurance, 1996. COMPASS, Systems Integrity. Software Safety. Process Security. Proceedings of the Eleventh Annual Conference on. 224-236.Google Scholar
Cross Ref
- Jeff Offutt and Jie Pan. 1997. Automatically detecting equivalent mutants and infeasible paths. Software Testing, Verification and Reliability 7, 3 (1997), 165-192.Google Scholar
Cross Ref
- Carlos Pacheco, Shuvendu Lahiri, Michael Ernst, and Thomas Ball. 2007. Feedback-directed random test generation. In Proceedings of the 29th International Conference on Software Engineering. 75-84. Google Scholar
Digital Library
- Mike Papadakis, Yue Jia, Mark Harman, and Yves Le Traon. 2015. Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple, Fast and Effective Equivalent Mutant Detection Technique. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. 936-946. Google Scholar
Digital Library
- Corina S. Pasareanu and Neha Rungta. 2010. Symbolic PathFinder: symbolic execution of Java bytecode. In Proceedings of the IEEE/ACM international conference on Automated software engineering. 179-180. Google Scholar
Digital Library
- David Schuler, Valentin Dallmeier, and Andreas Zeller. 2009. Efficient Mutation Testing by Checking Invariant Violations. In Proceedings of the Eighteenth International Symposium on Software Testing and Analysis. 69-80. Google Scholar
Digital Library
- David Schuler and Andreas Zeller. 2010. (Un-)Covering Equivalent Mutants. In Third International Conference on Software Testing, Verification and Validation. 45-54. Google Scholar
Digital Library
- David Schuler and Andreas Zeller. 2013. Covering and Uncovering Equivalent Mutants. Software Testing, Verification and Reliability 23, 5 (2013), 353-374.Google Scholar
Cross Ref
- Gustavo Soares, Rohit Gheyi, and Tiago Massoni. 2013. Automated behavioral testing of refactoring engines. IEEE Transactions on Software Engineering 39, 2 (2013), 147-162. Google Scholar
Digital Library
- Gustavo Soares, Rohit Gheyi, Dalton Serey, and Tiago Massoni. 2010. Making program refactoring safer. IEEE software 27, 4 (2010), 52-57. Google Scholar
Digital Library
- Jeffrey Voas and Gary McGraw. 1997. Software fault injection: inoculating programs against errors. John Wiley & Sons, Inc. Google Scholar
Digital Library
Index Terms
Avoiding useless mutants
Recommendations
Avoiding useless mutants
GPCE 2017: Proceedings of the 16th ACM SIGPLAN International Conference on Generative Programming: Concepts and ExperiencesMutation testing is a program-transformation technique that injects artificial bugs to check whether the existing test suite can detect them. However, the costs of using mutation testing are usually high, hindering its use in industry. Useless mutants (...
The Impact of Equivalent Mutants
ICSTW '09: Proceedings of the IEEE International Conference on Software Testing, Verification, and Validation WorkshopsIf a mutation is not killed by a test suite, this usuallymeans that the test suite is not adequate. However, itmay also be that the mutant keeps the program’s seman-tics unchanged—and thus cannot be detected by any test.We found such equivalent mutants ...
Mitigating the effects of equivalent mutants with mutant classification strategies
Mutation Testing has been shown to be a powerful technique in detecting software faults. Despite this advantage, in practice there is a need to deal with the equivalent mutants' problem. Automatically detecting equivalent mutants is an undecidable ...







Comments