Abstract
As concurrent programming becomes prevalent, software providers are investing in concurrency libraries to improve programmer productivity. Concurrency libraries improve productivity by hiding error-prone, low-level synchronization from programmers and providing higher-level concurrent abstractions. Testing such libraries is difficult, however, because concurrency failures often manifest only under particular scheduling circumstances. Current best testing practices are often inadequate: heuristic-guided fuzzing is not systematic, systematic schedule enumeration does not find bugs quickly, and stress testing is neither systematic nor fast.
To address these shortcomings, we propose a prioritized search technique called GAMBIT that combines the speed benefits of heuristic-guided fuzzing with the soundness, progress, and reproducibility guarantees of stateless model checking. GAMBIT combines known techniques such as partial-order reduction and preemption-bounding with a generalized best-first search frame- work that prioritizes schedules likely to expose bugs. We evaluate GAMBIT's effectiveness on newly released concurrency libraries for Microsoft's .NET framework. Our experiments show that GAMBIT finds bugs more quickly than prior stateless model checking techniques without compromising coverage guarantees or reproducibility.
- Concurrency and Coordination Runtime - http://msdn.microsoft.com/en-us/library/bb648752.aspx.Google Scholar
- E. M. Clarke and E. A. Emerson. Design and synthesis of synchronization skeletons using branching-time temporal logic. In Logic of Programs, pages 52--71, London, UK, 1981. Springer-Verlag. Google Scholar
Digital Library
- J. Duffy. A query language for data parallel programming: invited talk. In DAMP, page 50, 2007. Google Scholar
Digital Library
- M. B. Dwyer, S. Elbaum, S. Person, and R. Purandare. Parallel randomized state-space search. In Proceedings of the 29th international conference on Software Engineering (ICSE '07), pages 3--12, Washington, DC, USA, 2007. IEEE Computer Society. Google Scholar
Digital Library
- S. Edelkamp and S. Jabbar. Large-scale directed model checking ltl. In SPIN Workshop on Model Checking of Software, pages 1--18. Springer, 2006. Google Scholar
Digital Library
- S. Edelkamp, A. L. Lafuente, and S. Leue. Directed explicit model checking with HSF-SPIN. In SPIN Workshop on Model Checking of Software, pages 57--79. Springer-Verlag, 2001. Google Scholar
Digital Library
- O. Edelstein, E. Farchi, E. Goldin, Y. Nir, G. Ratsaby, and S. Ur. Framework for testing multi-threaded java programs. Concurrency and Computation: Practice and Experience, 15(3-5):485--499, 2003.Google Scholar
Cross Ref
- C. Flanagan and P. Godefroid. Dynamic partial-order reduction for model checking software. In 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL '05), pages 110--121, 2005. Google Scholar
Digital Library
- P. Godefroid. Partial-Order Methods for the Verification of Concurrent Systems: An Approach to the State-Explosion Problem. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 1996. Foreword By-Pierre Wolper. Google Scholar
Digital Library
- P. Godefroid.Model checking for programming languages using Verisoft. In POPL 97: Principles of Programming Languages, pages 174--186. ACM Press, 1997. Google Scholar
Digital Library
- P. Godefroid and S. Khurshid. Exploring very large state spaces using genetic algorithms. In In Tools and Algorithms for the Construction and Analysis of Systems, pages 266--280. Springer, 2002. Google Scholar
Digital Library
- P. Godefroid and P. Wolper. Using partial orders for the efficient verification of deadlock freedom and safety properties. In Computer Aided Verification (CAV '91), pages 332--342, 1992. Google Scholar
Digital Library
- A. Groce and W. Visser. Heuristic model checking for java programs. In Proceedings of the 9th International SPIN Workshop on Model Checking of Software, pages 242--245, London, UK, 2002. Springer-Verlag. Google Scholar
Digital Library
- G. Holzmann. The model checker SPIN. IEEE Transactions on Software Engineering, 23(5):279--295, May 1997. Google Scholar
Digital Library
- P. Joshi, M. Naik, C.-S. Park, and K. Sen. Calfuzzer: An extensible active testing framework for concurrent programs. In Proceedings of the 21st International Conference on Computer Aided Verification (CAV '09), pages 675--681, Berlin, Heidelberg, 2009. Springer-Verlag. Google Scholar
Digital Library
- A first look at jsr 166: Concurrency utilities http://today.java.net/pub/a/today/2004/03/01/jsr166.html.Google Scholar
- R. E. Korf, W. Zhang, I. Thayer, and H. Hohwald. Frontier search. Journal of the ACM, 52(5):715--748, 2005. Google Scholar
Digital Library
- D. Leijen, W. Schulte, and S. Burckhardt. The design of a task parallel library. In OOPSLA, page to appear, 2009. Google Scholar
Digital Library
- M. Musuvathi and S. Qadeer. Iterative context bounding for systematic testing of multithreaded programs. In Programming Language Design and Implementation (PLDI '07), pages 446--455, 2007. Google Scholar
Digital Library
- M. Musuvathi, S. Qadeer, T. Ball, G. Basler, A. Nainar, and I. Neamtiu. Finding and reproducing heisenbugs in concurrent programs. In 8th USENIX Symposium on Operating Systems Design and Implementation, 2009. Google Scholar
Digital Library
- W. T. Overman. Verification of concurrent systems: function and timing. PhD thesis, 1981. Google Scholar
Digital Library
- S. Park, S. Lu, and Y. Zhou. Ctrigger: exposing atomicity violation bugs from their hiding places. In ASPLOS, pages 25--36, 2009. Google Scholar
Digital Library
- J. Pearl. Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addison-Wesley, 1984. Google Scholar
Digital Library
- J.-P. Queille and J. Sifakis. Specification and verification of concurrent systems in cesar. In The Fifth International Symposium on Programming, pages 337--351, London, UK, 1982. Springer-Verlag. Google Scholar
Digital Library
- J. Reinders. Intel Threading Building Blocks : Outfitting C++ for Multi-core Processor Parallelism. O'Reilly, 2007. Google Scholar
Digital Library
- N. Rungta and E. G. Mercer. Guided model checking for programs with polymorphism.In Proceedings of the 2009 ACM SIGPLAN workshop on Partial evaluation and program manipulation (PEPM '09), pages 21--30, New York, NY, USA, 2009. ACM. Google Scholar
Digital Library
- S. J. Russell and P. Norvig. Artificial intelligence: a modern approach. Prentice Hall, 2nd edition, 2003. Google Scholar
Digital Library
- K. Sen. Effective random testing of concurrent programs. In ASE, pages 323--332, 2007. Google Scholar
Digital Library
- K. Sen. Race directed random testing of concurrent programs. In PLDI, pages 11--21, 2008. Google Scholar
Digital Library
- Optimize managed code for multi-core machines - http://msdn.microsoft.com/en-us/library/bb648752.aspx.Google Scholar
- A. Valmari. Stubborn sets for reduced state space generation. In The 10th International Conference on Applications and Theory of Petri Nets, pages 491--515, London, UK, 1991. Springer-Verlag. Google Scholar
Digital Library
- W. Visser, K. Havelund, G. Brat, and S. Park. Java PathFinder - second generation of a Java model checker. In Proceedings of Post- CAV Workshop on Advances in Verification, July 2000.Google Scholar
- C. H. Yang and D. L. Dill. Validation with guided search of the state space. In DAC '98, pages 599--604, 1998. Google Scholar
Digital Library
Index Terms
GAMBIT: effective unit testing for concurrency libraries
Recommendations
GAMBIT: effective unit testing for concurrency libraries
PPoPP '10: Proceedings of the 15th ACM SIGPLAN Symposium on Principles and Practice of Parallel ProgrammingAs concurrent programming becomes prevalent, software providers are investing in concurrency libraries to improve programmer productivity. Concurrency libraries improve productivity by hiding error-prone, low-level synchronization from programmers and ...
Iterative context bounding for systematic testing of multithreaded programs
PLDI '07: Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and ImplementationMultithreaded programs are difficult to get right because of unexpected interaction between concurrently executing threads. Traditional testing methods are inadequate for catching subtle concurrency errors which manifest themselves late in the ...
Iterative context bounding for systematic testing of multithreaded programs
Proceedings of the 2007 PLDI conferenceMultithreaded programs are difficult to get right because of unexpected interaction between concurrently executing threads. Traditional testing methods are inadequate for catching subtle concurrency errors which manifest themselves late in the ...







Comments