Abstract
Detecting concurrency-induced bugs in multithreaded libraries can be challenging due to the intricacies associated with their manifestation. This includes invocation of multiple methods, synthesis of inputs to the methods to reach the failing location, and crafting of thread interleavings that cause the erroneous behavior. Neither fuzzing-based testing techniques nor over-approximate static analyses are well positioned to detect such subtle defects while retaining high accuracy alongside satisfactory coverage.
In this paper, we propose a directed, iterative and scalable testing engine that combines the strengths of static and dynamic analysis to help synthesize concurrent executions to expose complex concurrency-induced bugs. Our engine accepts as input the library, its client (either sequential or concurrent) and a specification of correctness. Then, it iteratively refines the client to generate an execution that can break the input specification. Each step of the iterative process includes statically identifying sub-goals towards the goal of failing the specification, generating a plan toward meeting these goals, and merging of the paths traversed dynamically with the plan computed statically via constraint solving to generate a new client. The engine reports full reproduction scenarios, guaranteed to be true, for the bugs it finds.
We have created a prototype of our approach named MINION. We validated MINION by applying it to well-tested concurrent classes from popular Java libraries, including the latest versions of openjdk and google-guava. We were able to detect 31 real crashes across 10 classes in a total of 23 minutes, including previously unknown bugs. Comparison with three other tools reveals that combined, they report only 9 of the 31 crashes (and no other crashes beyond MINION). This is because several of these bugs manifest under deeply nested path conditions (observed maximum of 11), deep nesting of method invocations (observed maximum of 6) and multiple refinement iterations to generate the crash-inducing client.
- PushbackReader.java. http://grepcode.com/file/ repository.grepcode.com/java/root/jdk/ openjdk/8u40-b25/java/io/PushbackReader. java/.Google Scholar
- The Watson Libraries for Analysis. http://wala. sourceforge.net/wiki/index.php/Main_Page.Google Scholar
- A. Bessey, K. Block, B. Chelf, A. Chou, B. Fulton, S. Hallem, C. Henri-Gros, A. Kamsky, S. McPeak, and D. Engler. A few billion lines of code later: Using static analysis to find bugs in the real world. Commun. ACM, 53(2), 2010. Google Scholar
Digital Library
- S. Biswas, J. Huang, A. Sengupta, and M. D. Bond. Doublechecker: Efficient sound and precise atomicity checking. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, pages 28–39, 2014. Google Scholar
Digital Library
- C. Cadar, D. Dunbar, and D. Engler. Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI’08, pages 209–224, 2008. Google Scholar
Digital Library
- C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. Exe: Automatically generating inputs of death. ACM Trans. Inf. Syst. Secur., 12(2):10:1–10:38, Dec. 2008. Google Scholar
Digital Library
- S. Chandra, S. J. Fink, and M. Sridharan. Snugglebug: A powerful approach to weakest preconditions. In Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’09, pages 363–374, 2009. Google Scholar
Digital Library
- J.-D. Choi, K. Lee, A. Loginov, R. O’Callahan, V. Sarkar, and M. Sridharan. Efficient and precise datarace detection for multithreaded object-oriented programs. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation, PLDI ’02, pages 258–269, 2002. Google Scholar
Digital Library
- C. Csallner, Y. Smaragdakis, and T. Xie. Dsd-crasher: A hybrid analysis tool for bug finding. ACM Trans. Softw. Eng. Methodol., 17(2), May 2008. Google Scholar
Digital Library
- L. De Moura and N. Bjørner. Z3: An efficient smt solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS’08/ETAPS’08, pages 337–340, 2008. Google Scholar
Digital Library
- M. Eslamimehr and J. Palsberg. Sherlock: Scalable deadlock detection for concurrent programs. In Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2014, pages 353–365, 2014. Google Scholar
Digital Library
- M. Eslamimehr and J. Palsberg. Race directed scheduling of concurrent programs. In Proceedings of the 19th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP ’14, pages 301–314, 2014. Google Scholar
Digital Library
- C. Flanagan and S. N. Freund. Fasttrack: Efficient and precise dynamic race detection. In Proceedings of the 2009 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’09, 2009. Google Scholar
Digital Library
- C. Flanagan and P. Godefroid. Dynamic partial-order reduction for model checking software. In Proceedings of the 32Nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’05, pages 110–121, 2005. Google Scholar
Digital Library
- C. Flanagan, S. N. Freund, and J. Yi. Velodrome: A sound and complete dynamic atomicity checker for multithreaded programs. In Proceedings of the 2008 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’08, 2008. Google Scholar
Digital Library
- G. Fraser and A. Arcuri. Evosuite: Automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ESEC/FSE ’11, pages 416–419, 2011. Google Scholar
Digital Library
- P. Godefroid. Compositional dynamic test generation. In Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’07, pages 47–54, 2007. Google Scholar
Digital Library
- P. Godefroid, N. Klarlund, and K. Sen. Dart: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’05, 2005. Google Scholar
Digital Library
- J. Huang. Stateless model checking concurrent programs with maximal causality reduction. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2015, pages 165–174, 2015. Google Scholar
Digital Library
- J. Huang, C. Zhang, and J. Dolby. Clap: Recording local executions to reproduce concurrency failures. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’13, 2013. Google Scholar
Digital Library
- J. Huang, P. O. Meredith, and G. Rosu. Maximal sound predictive race detection with control flow abstraction. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, pages 337–348, 2014. Google Scholar
Digital Library
- V. Jagannath, M. Gligoric, D. Jin, Q. Luo, G. Rosu, and D. Marinov. Improved multithreaded unit testing. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ESEC/FSE ’11, pages 223–233, 2011. Google Scholar
Digital Library
- P. Joshi, C.-S. Park, K. Sen, and M. Naik. A randomized dynamic program analysis technique for detecting real deadlocks. In Proceedings of the 2009 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’09, 2009. Google Scholar
Digital Library
- B. Kasikci, C. Zamfir, and G. Candea. Data races vs. data race bugs: telling the difference with portend. In Proceedings of the 17th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2012, London, UK, March 3-7, 2012, pages 185–198, 2012. Google Scholar
Digital Library
- P. Liu, X. Zhang, O. Tripp, and Y. Zheng. Light: Replay via tightly bounded recording. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2015, pages 55–64, 2015. Google Scholar
Digital Library
- B. Livshits, M. Sridharan, Y. Smaragdakis, O. Lhoták, J. N. Amaral, B.-Y. E. Chang, S. Z. Guyer, U. P. Khedker, A. Møller, and D. Vardoulakis. In defense of soundiness: A manifesto. Commun. ACM, 58(2):44–46, Jan. 2015. Google Scholar
Digital Library
- S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from mistakes: A comprehensive study on real world concurrency bug characteristics. In Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIII, pages 329–339, 2008. Google Scholar
Digital Library
- N. Machado, B. Lucia, and L. Rodrigues. Concurrency debugging with differential schedule projections. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2015, pages 586– 595, 2015. Google Scholar
Digital Library
- N. Machado, B. Lucia, and L. Rodrigues. Concurrency debugging with differential schedule projections. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2015, pages 586– 595, 2015. Google Scholar
Digital Library
- N. Machado, B. Lucia, and L. E. T. Rodrigues. Productionguided concurrency debugging. In Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2016, Barcelona, Spain, March 12-16, 2016, pages 29:1–29:12, 2016. Google Scholar
Digital Library
- S. McPeak, C.-H. Gros, and M. K. Ramanathan. Scalable and incremental software bug detection. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013. Google Scholar
Digital Library
- S. S. Muchnick. Advanced Compiler Design and Implementation. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1997. ISBN 1-55860-320-4. Google Scholar
Digital Library
- M. Musuvathi and S. Qadeer. Logic-Based Program Synthesis and Transformation: 16th International Symposium, LOPSTR 2006, Venice, Italy, July 12-14, 2006, Revised Selected Papers, chapter CHESS: Systematic Stress Testing of Concurrent Software, pages 15–16. Springer Berlin Heidelberg, Berlin, Heidelberg, 2007. Google Scholar
Digital Library
- S. Nagarakatte, S. Burckhardt, M. M. Martin, and M. Musuvathi. Multicore acceleration of priority-based schedulers for concurrency bug detection. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’12, 2012. Google Scholar
Digital Library
- S. Narayanasamy, Z. Wang, J. Tigani, A. Edwards, and B. Calder. Automatically classifying benign and harmful data races using replay analysis. In Proceedings of the 2007 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’07, 2007. Google Scholar
Digital Library
- C. Pacheco, S. K. Lahiri, M. D. Ernst, and T. Ball. Feedbackdirected random test generation. In Proceedings of the 29th International Conference on Software Engineering, ICSE ’07, pages 75–84, 2007. ISBN 0-7695-2828-7. Google Scholar
Digital Library
- S. Park, S. Lu, and Y. Zhou. Ctrigger: Exposing atomicity violation bugs from their hiding places. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIV, 2009. Google Scholar
Digital Library
- M. Pradel and T. R. Gross. Fully automatic and precise detection of thread safety violations. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’12, 2012. Google Scholar
Digital Library
- D. Prountzos, R. Manevich, and K. Pingali. Synthesizing parallel graph programs via automated planning. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2015, pages 533– 544, 2015. Google Scholar
Digital Library
- W. Pugh and N. Ayewah. Unit testing concurrent software. In Proceedings of the Twenty-second IEEE/ACM International Conference on Automated Software Engineering, ASE ’07, pages 513–516, 2007. Google Scholar
Digital Library
- M. Samak and M. K. Ramanathan. Multithreaded test synthesis for deadlock detection. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’14, pages 473–489, 2014. Google Scholar
Digital Library
- M. Samak and M. K. Ramanathan. Trace driven dynamic deadlock detection and reproduction. In Proceedings of the 2014 ACM SIGPLAN Conference on Principles and Practices of Parallel Programming, PPoPP ’14, 2014. Google Scholar
Digital Library
- M. Samak and M. K. Ramanathan. Synthesizing tests for detecting atomicity violations. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, pages 131–142, 2015. Google Scholar
Digital Library
- M. Samak, M. K. Ramanathan, and S. Jagannathan. Synthesizing racy tests. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2015, pages 175–185, 2015. Google Scholar
Digital Library
- K. Sen. Race directed random testing of concurrent programs. In Proceedings of the 2008 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’08, pages 11–21, 2008. Google Scholar
Digital Library
- Y. Smaragdakis, J. Evans, C. Sadowski, J. Yi, and C. Flanagan. Sound predictive race detection in polynomial time. In Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’12, pages 387–400, 2012. Google Scholar
Digital Library
- S. Steenbuck and G. Fraser. Generating unit tests for concurrent classes. In Proceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, ICST ’13, pages 144–153, 2013. Google Scholar
Digital Library
- B. Swarnendu, Z. Minjia, B. Michael, and L. Brandon. In Proceedings of the 2015 ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’15, 2015.Google Scholar
- S. Thummalapenta, T. Xie, N. Tillmann, J. de Halleux, and Z. Su. Synthesizing method sequences for high-coverage testing. In Proceedings of the 2011 ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’11. Google Scholar
Digital Library
- O. Tripp, O. Weisman, and L. Guy. Finding your way in the testing jungle: A learning approach to web security testing. In Proceedings of the 2013 International Symposium on Software Testing and Analysis, pages 347–357, 2013. Google Scholar
Digital Library
- R. Vallee-Rai, E. Gagnon, L. Hendren, P. Lam, P. Pominville, and V. Sundaresan. Optimizing java bytecode using the soot framework: Is it feasible? In In International Conference on Compiler Construction, LNCS 1781, pages 18–34, 2000. Google Scholar
Digital Library
- C. Wang, S. Kundu, M. Ganai, and A. Gupta. Symbolic predictive analysis for concurrent programs. In Proceedings of the 2Nd World Congress on Formal Methods, FM ’09, pages 256–272, 2009. Google Scholar
Digital Library
- C. Zamfir and G. Candea. Execution synthesis: a technique for automated software debugging. In European Conference on Computer Systems, Proceedings of the 5th European conference on Computer systems, EuroSys 2010, Paris, France, April 13-16, 2010, pages 321–334, 2010. Google Scholar
Digital Library
Index Terms
Directed synthesis of failing concurrent executions
Recommendations
Directed synthesis of failing concurrent executions
OOPSLA 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and ApplicationsDetecting concurrency-induced bugs in multithreaded libraries can be challenging due to the intricacies associated with their manifestation. This includes invocation of multiple methods, synthesis of inputs to the methods to reach the failing location, ...
Multithreaded test synthesis for deadlock detection
OOPSLA '14Designing and implementing thread-safe multithreaded libraries can be a daunting task as developers of these libraries need to ensure that their implementations are free from concurrency bugs, including deadlocks. The usual practice involves employing ...
Multithreaded test synthesis for deadlock detection
OOPSLA '14: Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & ApplicationsDesigning and implementing thread-safe multithreaded libraries can be a daunting task as developers of these libraries need to ensure that their implementations are free from concurrency bugs, including deadlocks. The usual practice involves employing ...







Comments