Abstract
To report a compiler bug, one must often find a small test case that triggers the bug. The existing approach to automated test-case reduction, delta debugging, works by removing substrings of the original input; the result is a concatenation of substrings that delta cannot remove. We have found this approach less than ideal for reducing C programs because it typically yields test cases that are too large or even invalid (relying on undefined behavior). To obtain small and valid test cases consistently, we designed and implemented three new, domain-specific test-case reducers. The best of these is based on a novel framework in which a generic fixpoint computation invokes modular transformations that perform reduction operations. This reducer produces outputs that are, on average, more than 25 times smaller than those produced by our other reducers or by the existing reducer that is most commonly used by compiler developers. We conclude that effective program reduction requires more than straightforward delta debugging.
- Bruno Blanchet, Patrick Cousot, Radhia Cousot, Jérôme Feret, Laurent Mauborgne, Antoine Miné, David Monniaux, and Xavier Rival. A static analyzer for large safety-critical software. In Proc. of the ACM SIGPLAN 2003 Conf. on Programming Language Design and Implementation (PLDI), pages 196--207, San Diego, CA, June 2003. Google Scholar
Digital Library
- Géraud Canet, Pascal Cuoq, and Benjamin Monate. A value analysis for C programs. In Proc. of the 9th IEEE Intl. Working Conf. on Source Code Analysis and Manipulation, pages 123--124, Edmonton, Alberta, Canada, September 2009. Google Scholar
Digital Library
- Jacqueline M. Caron and Peter A. Darnell. Bugfind: A tool for debugging optimizing compilers. SIGPLAN Notices, 25(1):17--22, January 1990. Google Scholar
Digital Library
- Pascal Cuoq, Benjamin Monate, Anne Pacalet, Virgile Prevosto, John Regehr, Boris Yakobowski, and Xuejun Yang. Testing static analyzers with randomly generated programs. In Proc. of the 4th NASA Formal Methods Symposium (NFM 2012), Norfolk, VA, April 2012. Google Scholar
Digital Library
- Pascal Cuoq, Julien Signoles, Patrick Baudin, Richard Bonichon, Géraud Canet, Loıc Correnson, Benjamin Monate, Virgile Prevosto, and Armand Puccetti. Experience report: OCaml for an industrial-strength static analysis framework. In Proc. of the 14th ACM SIGPLAN Intl. Conf. on Functional Programming (ICFP), pages 281--286, Edinburgh, Scotland, 2009. Google Scholar
Digital Library
- Chucky Ellison and Grigore Roşu. An executable formal semantics of C with applications. In Proc. of the 39th Symp. on Principles of Programming Languages (POPL), pages 533--544, Philadelphia, PA, January 2012. Google Scholar
Digital Library
- International Organization for Standardization. ISO/IEC 9899:TC3: Programming Languages-C, 2007. http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf.Google Scholar
- Rajeev Joshi, Greg Nelson, and Yunhong Zhou. Denali: A practical algorithm for generating optimal code. ACM Transactions on Programming Languages and Systems, 28(6):967--989, November 2006. Google Scholar
Digital Library
- Andreas Leitner, Manuel Oriol, Andreas Zeller, Ilinca Ciupa, and Bertrand Meyer. Efficient unit test case minimization. In Proc. of the 22nd Intl. Conf. on Automated Software Engineering (ASE), pages 417--420, Atlanta, GA, November 2007. Google Scholar
Digital Library
- MathWorks. Polyspace server 8.1 for C/C, product brochure, 2010. http://www.mathworks.com/products/datasheets/pdf/polyspace-server-for-c-c.pdf.Google Scholar
- William M. McKeeman. Differential testing for software. Digital Technical Journal, 10(1):100--107, December 1998.Google Scholar
- Scott McPeak and Daniel S. Wilkerson. Delta, 2003. http://delta.tigris.org/.Google Scholar
- Ghassan Misherghi and Zhendong Su. HDD: Hierarchical delta debugging. In Proc. of the 28th Intl. Conf. on Software Engineering (ICSE), pages 142--151, Shanghai, China, May 2006. Google Scholar
Digital Library
- James W. Moore. ISO/IEC JTC 1/SC 22/WG 23: Programming language vulnerabilities. http://grouper.ieee.org/groups/plv/.Google Scholar
- David B. Whalley. Automatic isolation of compiler errors. ACM Transactions on Programming Languages and Systems, 16(5):1648--1659, September 1994. Google Scholar
Digital Library
- Brian White, Jay Lepreau, Leigh Stoller, Robert Ricci, Shashi Guruprasad, Mac Newbold, Mike Hibler, Chad Barb, and Abhijeet Joglekar. An integrated experimental environment for distributed systems and networks. In Proc. of the 5th Symp. on Operating Systems Design and Implementation (OSDI), pages 255--270, Boston, MA, December 2002. Google Scholar
Digital Library
- Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. Finding and understanding bugs in C compilers. In Proc. of the ACM SIGPLAN 2011 Conf. on Programming Language Design and Implementation (PLDI), pages 283--294, San Jose, CA, June 2011. Google Scholar
Digital Library
- Andreas Zeller and Ralf Hildebrandt. Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering, 28(2):183--200, February 2002. Google Scholar
Digital Library
Index Terms
Test-case reduction for C compiler bugs
Recommendations
Test-case reduction for C compiler bugs
PLDI '12: Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and ImplementationTo report a compiler bug, one must often find a small test case that triggers the bug. The existing approach to automated test-case reduction, delta debugging, works by removing substrings of the original input; the result is a concatenation of ...
Random testing for C and C++ compilers with YARPGen
Compilers should not crash and they should not miscompile applications. Random testing is an effective method for finding compiler bugs that have escaped other kinds of testing. This paper presents Yet Another Random Program Generator (YARPGen), a ...
Finding and understanding bugs in C compilers
PLDI '11Compilers should be correct. To improve the quality of C compilers, we created Csmith, a randomized test-case generation tool, and spent three years using it to find compiler bugs. During this period we reported more than 325 previously unknown bugs to ...







Comments