skip to main content
research-article

GAMBIT: effective unit testing for concurrency libraries

Published:09 January 2010Publication History
Skip Abstract Section

Abstract

As concurrent programming becomes prevalent, software providers are investing in concurrency libraries to improve programmer productivity. Concurrency libraries improve productivity by hiding error-prone, low-level synchronization from programmers and providing higher-level concurrent abstractions. Testing such libraries is difficult, however, because concurrency failures often manifest only under particular scheduling circumstances. Current best testing practices are often inadequate: heuristic-guided fuzzing is not systematic, systematic schedule enumeration does not find bugs quickly, and stress testing is neither systematic nor fast.

To address these shortcomings, we propose a prioritized search technique called GAMBIT that combines the speed benefits of heuristic-guided fuzzing with the soundness, progress, and reproducibility guarantees of stateless model checking. GAMBIT combines known techniques such as partial-order reduction and preemption-bounding with a generalized best-first search frame- work that prioritizes schedules likely to expose bugs. We evaluate GAMBIT's effectiveness on newly released concurrency libraries for Microsoft's .NET framework. Our experiments show that GAMBIT finds bugs more quickly than prior stateless model checking techniques without compromising coverage guarantees or reproducibility.

References

  1. Concurrency and Coordination Runtime - http://msdn.microsoft.com/en-us/library/bb648752.aspx.Google ScholarGoogle Scholar
  2. E. M. Clarke and E. A. Emerson. Design and synthesis of synchronization skeletons using branching-time temporal logic. In Logic of Programs, pages 52--71, London, UK, 1981. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. J. Duffy. A query language for data parallel programming: invited talk. In DAMP, page 50, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. M. B. Dwyer, S. Elbaum, S. Person, and R. Purandare. Parallel randomized state-space search. In Proceedings of the 29th international conference on Software Engineering (ICSE '07), pages 3--12, Washington, DC, USA, 2007. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. S. Edelkamp and S. Jabbar. Large-scale directed model checking ltl. In SPIN Workshop on Model Checking of Software, pages 1--18. Springer, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. S. Edelkamp, A. L. Lafuente, and S. Leue. Directed explicit model checking with HSF-SPIN. In SPIN Workshop on Model Checking of Software, pages 57--79. Springer-Verlag, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. O. Edelstein, E. Farchi, E. Goldin, Y. Nir, G. Ratsaby, and S. Ur. Framework for testing multi-threaded java programs. Concurrency and Computation: Practice and Experience, 15(3-5):485--499, 2003.Google ScholarGoogle ScholarCross RefCross Ref
  8. C. Flanagan and P. Godefroid. Dynamic partial-order reduction for model checking software. In 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL '05), pages 110--121, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. P. Godefroid. Partial-Order Methods for the Verification of Concurrent Systems: An Approach to the State-Explosion Problem. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 1996. Foreword By-Pierre Wolper. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. P. Godefroid.Model checking for programming languages using Verisoft. In POPL 97: Principles of Programming Languages, pages 174--186. ACM Press, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. P. Godefroid and S. Khurshid. Exploring very large state spaces using genetic algorithms. In In Tools and Algorithms for the Construction and Analysis of Systems, pages 266--280. Springer, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. P. Godefroid and P. Wolper. Using partial orders for the efficient verification of deadlock freedom and safety properties. In Computer Aided Verification (CAV '91), pages 332--342, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. A. Groce and W. Visser. Heuristic model checking for java programs. In Proceedings of the 9th International SPIN Workshop on Model Checking of Software, pages 242--245, London, UK, 2002. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. G. Holzmann. The model checker SPIN. IEEE Transactions on Software Engineering, 23(5):279--295, May 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. P. Joshi, M. Naik, C.-S. Park, and K. Sen. Calfuzzer: An extensible active testing framework for concurrent programs. In Proceedings of the 21st International Conference on Computer Aided Verification (CAV '09), pages 675--681, Berlin, Heidelberg, 2009. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A first look at jsr 166: Concurrency utilities http://today.java.net/pub/a/today/2004/03/01/jsr166.html.Google ScholarGoogle Scholar
  17. R. E. Korf, W. Zhang, I. Thayer, and H. Hohwald. Frontier search. Journal of the ACM, 52(5):715--748, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. D. Leijen, W. Schulte, and S. Burckhardt. The design of a task parallel library. In OOPSLA, page to appear, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. M. Musuvathi and S. Qadeer. Iterative context bounding for systematic testing of multithreaded programs. In Programming Language Design and Implementation (PLDI '07), pages 446--455, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. Musuvathi, S. Qadeer, T. Ball, G. Basler, A. Nainar, and I. Neamtiu. Finding and reproducing heisenbugs in concurrent programs. In 8th USENIX Symposium on Operating Systems Design and Implementation, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. W. T. Overman. Verification of concurrent systems: function and timing. PhD thesis, 1981. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. S. Park, S. Lu, and Y. Zhou. Ctrigger: exposing atomicity violation bugs from their hiding places. In ASPLOS, pages 25--36, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. J. Pearl. Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addison-Wesley, 1984. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J.-P. Queille and J. Sifakis. Specification and verification of concurrent systems in cesar. In The Fifth International Symposium on Programming, pages 337--351, London, UK, 1982. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. J. Reinders. Intel Threading Building Blocks : Outfitting C++ for Multi-core Processor Parallelism. O'Reilly, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. N. Rungta and E. G. Mercer. Guided model checking for programs with polymorphism.In Proceedings of the 2009 ACM SIGPLAN workshop on Partial evaluation and program manipulation (PEPM '09), pages 21--30, New York, NY, USA, 2009. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. S. J. Russell and P. Norvig. Artificial intelligence: a modern approach. Prentice Hall, 2nd edition, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. K. Sen. Effective random testing of concurrent programs. In ASE, pages 323--332, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. K. Sen. Race directed random testing of concurrent programs. In PLDI, pages 11--21, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Optimize managed code for multi-core machines - http://msdn.microsoft.com/en-us/library/bb648752.aspx.Google ScholarGoogle Scholar
  31. A. Valmari. Stubborn sets for reduced state space generation. In The 10th International Conference on Applications and Theory of Petri Nets, pages 491--515, London, UK, 1991. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. W. Visser, K. Havelund, G. Brat, and S. Park. Java PathFinder - second generation of a Java model checker. In Proceedings of Post- CAV Workshop on Advances in Verification, July 2000.Google ScholarGoogle Scholar
  33. C. H. Yang and D. L. Dill. Validation with guided search of the state space. In DAC '98, pages 599--604, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. GAMBIT: effective unit testing for concurrency libraries

                      Recommendations

                      Comments

                      Login options

                      Check if you have access through your login credentials or your institution to get full access on this article.

                      Sign in

                      Full Access

                      PDF Format

                      View or Download as a PDF file.

                      PDF

                      eReader

                      View online with eReader.

                      eReader
                      About Cookies On This Site

                      We use cookies to ensure that we give you the best experience on our website.

                      Learn more

                      Got it!