Abstract
Bounded model checking based debugging solutions search for mutations of program expressions that produce the expected output for a currently failing test. However, the current localization tools are not regression aware: they do not use information from the passing tests in their localization formula. On the other hand, the current repair tools attempt to guarantee regression freedom: when provided with a set of passing tests, they guarantee that none of these tests can break due to the suggested repair patch, thereby constructing a large repair formula.
In this paper, we propose regression awareness as a means to improve the quality of localization and to scale repair. To enable regression awareness, we summarize the proof of correctness of each passing test by computing Craig Interpolants over a symbolic encoding of the passing execution, and use these summaries as additional soft constraints while synthesizing altered executions corresponding to failing tests. Intuitively, these additional constraints act as roadblocks, thereby discouraging executions that may damage the proof of a passing test. We use a partial MAXSAT solver to relax the proofs in a systematic way, and use a ranking function that penalizes mutations that damage the existing proofs.
We have implemented our algorithms into a tool, TINTIN, that enables regression aware localization and repair. For localizations, our strategy is effective in extracting a superior ranking of suspicious locations: on a set of 52 different versions across 12 different programs spanning three benchmark suites, TINTIN achieves a saving of developer effort by almost 45% (in terms of the locations that must be examined by a developer to reach the ground-truth repair) in the worst case and 27% in the average case over existing techniques. For automated repairs, on our set of benchmarks, TINTIN achieves a 2.3X speedup over existing techniques without sacrificing much on the ranking of the repair patches: the ground-truth repair appears as the topmost suggestion in more than 70% of our benchmarks.
- Software-artifact infrastructure repository (SIR). http://sir.unl.edu/portal/index.php.Google Scholar
- Aws Albarghouthi, Arie Gurfinkel, Yi Li, Sagar Chaki, and Marsha Chechik. UFO: Verification with Interpolants and Abstract Interpretation. In Proceedings of the 19th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS’13, pages 637–640, Berlin, Heidelberg, 2013. Springer-Verlag. Google Scholar
Digital Library
- Aws Albarghouthi, Yi Li, Arie Gurfinkel, and Marsha Chechik. UFO: A Framework for Abstractionand Interpolation-based Software Verification. In Proceedings of the 24th International Conference on Computer Aided Verification, CAV’12, pages 672–678, Berlin, Heidelberg, 2012. Springer-Verlag. Google Scholar
Digital Library
- Aws Albarghouthi and Kenneth L. McMillan. Beautiful Interpolants. In Proceedings of the 25th International Conference on Computer Aided Verification, CAV’13, pages 313– 329, Berlin, Heidelberg, 2013. Springer-Verlag.Google Scholar
- Thomas Ball, Rupak Majumdar, Todd Millstein, and Sriram K. Rajamani. Automatic Predicate Abstraction of C Programs. In Proceedings of the ACM SIGPLAN 2001 Conference on Programming Language Design and Implementation, PLDI ’01, pages 203–213, New York, NY, USA, 2001. ACM. Google Scholar
Digital Library
- Thomas Ball, Mayur Naik, and Sriram K. Rajamani. From Symptom to Cause: Localizing Errors in Counterexample Traces. In Proceedings of the 30th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’03, pages 97–105, New York, NY, USA, 2003. ACM. Google Scholar
Digital Library
- Dirk Beyer. Software Verification and Verifiable Witnesses - (Report on SV-COMP 2016), 2016. (accessed in Jan 2016) https://github.com/sosy-lab/sv-benchmarks.Google Scholar
- Nikolaj Bjørner, Arie Gurfinkel, Ken McMillan, and Andrey Rybalchenko. Horn clause solvers for program verification. In Fields of Logic and Computation II, pages 24–51. Springer, 2015.Google Scholar
- Cristian Cadar, Daniel Dunbar, and Dawson Engler. KLEE: Unassisted and Automatic Generation of High-coverage Tests for Complex Systems Programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI’08, pages 209–224, Berkeley, CA, USA, 2008. USENIX Association. Google Scholar
Digital Library
- Satish Chandra, Emina Torlak, Shaon Barman, and Rastislav Bodik. Angelic Debugging. In Proceedings of the 33rd International Conference on Software Engineering, ICSE ’11, pages 121–130, New York, NY, USA, 2011. ACM. Google Scholar
Digital Library
- Alessandro Cimatti, Alberto Griggio, Bastiaan Joost Schaafsma, and Roberto Sebastiani. The MathSAT5 SMT Solver. In Proceedings of the 19th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS’13, pages 93–107, Berlin, Heidelberg, 2013. Springer-Verlag. Google Scholar
Digital Library
- Edmund Clarke, Daniel Kroening, and Flavio Lerda. A tool for checking ANSI-C programs. In Tools and Algorithms for the Construction and Analysis of Systems, pages 168–176. Springer, 2004.Google Scholar
- Patrick Cousot and Radhia Cousot. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In Proceedings of the 4th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages, POPL ’77, pages 238–252, New York, NY, USA, 1977. ACM. Google Scholar
Digital Library
- William Craig. Three uses of the Herbrand-Gentzen theorem in relating model theory and proof theory. The Journal of Symbolic Logic, 22(03):269–285, 1957.Google Scholar
Cross Ref
- Hyunsook Do, Sebastian Elbaum, and Gregg Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Softw. Engg., 10(4):405–435, October 2005. Google Scholar
Digital Library
- Zhaohui Fu and Sharad Malik. On Solving the Partial MAXSAT Problem. In Proceedings of the 9th International Conference on Theory and Applications of Satisfiability Testing, SAT’06, pages 252–265, Berlin, Heidelberg, 2006. Springer-Verlag. Google Scholar
Digital Library
- Patrice Godefroid, Nils Klarlund, and Koushik Sen. DART: Directed Automated Random Testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’05, pages 213–223, New York, NY, USA, 2005. ACM. Google Scholar
Digital Library
- Alex Groce, Sagar Chaki, Daniel Kroening, and Ofer Strichman. Error Explanation with Distance Metrics. Int. J. Softw. Tools Technol. Transf., 8(3):229–247, June 2006.Google Scholar
Cross Ref
- Joxan Jaffar, Vijayaraghavan Murali, and Jorge A. Navas. Boosting Concolic Testing via Interpolation. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013, pages 48–58, New York, NY, USA, 2013. ACM. Google Scholar
Digital Library
- Joxan Jaffar, Vijayaraghavan Murali, Jorge A. Navas, and Andrew E. Santosa. TRACER: A Symbolic Execution Tool for Verification. In Proceedings of the 24th International Conference on Computer Aided Verification, CAV’12, pages 758–766, Berlin, Heidelberg, 2012. Springer-Verlag. Google Scholar
Digital Library
- Joxan Jaffar, Jorge A. Navas, and Andrew E. Santosa. Unbounded Symbolic Execution for Program Verification. In Proceedings of the Second International Conference on Runtime Verification, RV’11, pages 396–411, Berlin, Heidelberg, 2012. Springer-Verlag. Google Scholar
Digital Library
- Susmit Jha, Sumit Gulwani, Sanjit A. Seshia, and Ashish Tiwari. Oracle-guided Component-based Program Synthesis. In Proceedings of the 32Nd ACM/IEEE International Conference on Software Engineering - Volume 1, ICSE ’10, pages 215–224, New York, NY, USA, 2010. ACM. Google Scholar
Digital Library
- Manu Jose and Rupak Majumdar. Cause Clue Clauses: Error Localization Using Maximum Satisfiability. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’11, pages 437– 446, New York, NY, USA, 2011. ACM. Google Scholar
Digital Library
- Claire Le Goues, ThanhVu Nguyen, Stephanie Forrest, and Westley Weimer. GenProg: A Generic Method for Automatic Software Repair. IEEE Trans. Softw. Eng., 38(1):54–72, January 2012. Google Scholar
Digital Library
- Chu Min Li and Felip Manya. MaxSAT, Hard and Soft Constraints. Handbook of satisfiability, 185:613–631, 2009.Google Scholar
- Ben Liblit, Mayur Naik, Alice X. Zheng, Alex Aiken, and Michael I. Jordan. Scalable Statistical Bug Isolation. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’05, pages 15–26, New York, NY, USA, 2005. ACM. Google Scholar
Digital Library
- Chao Liu, Xifeng Yan, Long Fei, Jiawei Han, and Samuel P. Midkiff. SOBER: Statistical Model-based Bug Localization. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-13, pages 286–295, New York, NY, USA, 2005. ACM. Google Scholar
Digital Library
- Yongmei Liu and Bing Li. Automated Program Debugging via Multiple Predicate Switching. In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, AAAI’10, pages 327–332. AAAI Press, 2010. Google Scholar
Digital Library
- Fan Long and Martin Rinard. Staged Program Repair with Condition Synthesis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, pages 166–178, New York, NY, USA, 2015. ACM. Google Scholar
Digital Library
- Joao Marques-Silva. Minimal Unsatisfiability: Models, Algorithms and Applications (Invited Paper). In Proceedings of the 2010 40th IEEE International Symposium on Multiple-Valued Logic, ISMVL ’10, pages 9–14, Washington, DC, USA, 2010. IEEE Computer Society. Google Scholar
Digital Library
- Joao Marques-Silva and Jordi Planes. Algorithms for Maximum Satisfiability Using Unsatisfiable Cores. In Proceedings of the Conference on Design, Automation and Test in Europe, DATE ’08, pages 408–413, New York, NY, USA, 2008. ACM. Google Scholar
Digital Library
- K. L. McMillan. Applications of Craig Interpolants in Model Checking. In Proceedings of the 11th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS’05, pages 1–12, Berlin, Heidelberg, 2005. Springer-Verlag. Google Scholar
Digital Library
- Kenneth L. McMillan. Lazy Abstraction with Interpolants. In Proceedings of the 18th International Conference on Computer Aided Verification, CAV’06, pages 123–136, Berlin, Heidelberg, 2006. Springer-Verlag. Google Scholar
Digital Library
- Kenneth L. McMillan. Lazy Annotation for Program Testing and Verification. In Proceedings of the 22Nd International Conference on Computer Aided Verification, CAV’10, pages 104–118, Berlin, Heidelberg, 2010. Springer-Verlag. Google Scholar
Digital Library
- Kenneth L. Mcmillan. Lazy Annotation Revisited. In Proceedings of the 16th International Conference on Computer Aided Verification - Volume 8559, pages 243–259, New York, NY, USA, 2014. Springer-Verlag New York, Inc. Google Scholar
Digital Library
- Sergey Mechtaev, Jooyong Yi, and Abhik Roychoudhury. DirectFix: Looking for Simple Program Repairs. In Proceedings of the 37th International Conference on Software Engineering - Volume 1, ICSE ’15, pages 448–458, Piscataway, NJ, USA, 2015. Google Scholar
Digital Library
- Sergey Mechtaev, Jooyong Yi, and Abhik Roychoudhury. Angelix: Scalable multiline program patch synthesis via symbolic analysis. In Proceedings of the 38th International Conference on Software Engineering, ICSE ’16, pages 691–701, New York, NY, USA, 2016. ACM. Google Scholar
Digital Library
- Varun Modi, Subhajit Roy, and Sanjeev K. Aggarwal. Exploring Program Phases for Statistical Bug Localization. In Proceedings of the 11th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, PASTE ’13, pages 33–40, New York, NY, USA, 2013. ACM. Google Scholar
Digital Library
- Martin Monperrus. A Critical Review of ”Automatic Patch Generation Learned from Human-written Patches”: Essay on the Problem Statement and the Evaluation of Automatic Software Repair. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, pages 234–242, New York, NY, USA, 2014. ACM. Google Scholar
Digital Library
- Hoang Duong Thien Nguyen, Dawei Qi, Abhik Roychoudhury, and Satish Chandra. SemFix: Program Repair via Semantic Analysis. In Proceedings of the 2013 International Conference on Software Engineering, ICSE ’13, pages 772– 781, Piscataway, NJ, USA, 2013. IEEE Press. Google Scholar
Digital Library
- ThanhVu Nguyen, Westley Weimer, Claire Le Goues, and Stephanie Forrest. Using execution paths to evolve software patches. In Software Testing, Verification and Validation Workshops, 2009. ICSTW’09. International Conference on, pages 152–153. IEEE, 2009. Google Scholar
Digital Library
- Manos Renieres and Steven P Reiss. Fault localization with nearest neighbor queries. In Automated Software Engineering, 2003. Proceedings. 18th IEEE International Conference on, pages 30–39. IEEE, 2003.Google Scholar
- Hesam Samimi, Max Schäfer, Shay Artzi, Todd D. Millstein, Frank Tip, and Laurie J. Hendren. Automated repair of HTML generation errors in PHP applications using string constraint solving. In 34th International Conference on Software Engineering, ICSE 2012, June 2-9, 2012, Zurich, Switzerland, pages 277–287, 2012. Google Scholar
Digital Library
- Ondrej Sery, Grigory Fedyukovich, and Natasha Sharygina. FunFrog: Bounded Model Checking with Interpolation-Based Function Summarization, pages 203–207. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012. Google Scholar
Digital Library
- Frank Tip. A survey of program slicing techniques. J. Prog. Lang., 3(3), 1995.Google Scholar
- Wei Wang and Clark Barrett. Cascade 2016 - (Competition Contribution), 2016.Google Scholar
- (accessed in Feb 2016) http://cascade.cims.nyu.edu/vmcai.html.Google Scholar
- Westley Weimer. Patches As Better Bug Reports. In Proceedings of the 5th International Conference on Generative Programming and Component Engineering, GPCE ’06, pages 181–190, New York, NY, USA, 2006. ACM. Google Scholar
Digital Library
- Westley Weimer, ThanhVu Nguyen, Claire Le Goues, and Stephanie Forrest. Automatically Finding Patches Using Genetic Programming. In Proceedings of the 31st International Conference on Software Engineering, ICSE ’09, pages 364– 374, Washington, DC, USA, 2009. IEEE Computer Society. Google Scholar
Digital Library
- Andreas Zeller and Ralf Hildebrandt. Simplifying and Isolating Failure-Inducing Input. IEEE Trans. Softw. Eng., 28(2):183–200, February 2002. Google Scholar
Digital Library
Index Terms
To be precise: regression aware debugging
Recommendations
To be precise: regression aware debugging
OOPSLA 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and ApplicationsBounded model checking based debugging solutions search for mutations of program expressions that produce the expected output for a currently failing test. However, the current localization tools are not regression aware: they do not use information ...
Regression aware debugging for mobile applications
Mobile! 2016: Proceedings of the 1st International Workshop on Mobile DevelopmentRegression-aware fault localization attempts to rank suspicious statements in a manner such that potential regression inducing suggestions are ranked low. The algorithm extracts the proof of correctness of all the correct executions in the form of ...
A similarity-aware approach to testing based fault localization
ASE '05: Proceedings of the 20th IEEE/ACM International Conference on Automated Software EngineeringDebugging is a time-consuming task in software development and maintenance. To accelerate this task, several approaches have been proposed to automate fault localization. In particular, testing based fault localization (TBFL), which utilizes the testing ...







Comments