skip to main content
research-article

Detecting problematic message sequences and frequencies in distributed systems

Published:19 October 2012Publication History
Skip Abstract Section

Abstract

Testing the components of a distributed system is challenging as it requires consideration of not just the state of a component, but also the sequence of messages it may receive from the rest of the system or the environment. Such messages may vary in type and content, and more particularly, in the frequency at which they are generated. All of these factors, in the right combination, may lead to faulty behavior. In this paper we present an approach to address these challenges by systematically analyzing a component in a distributed system to identify specific message sequences and frequencies at which a failure can occur. At the core of the analysis is the generation of a test driver that defines the space of message sequences to be generated, the exploration of that space through the use of dynamic symbolic execution, and the timing and analysis of the generated tests to identify problematic frequencies. We implemented our approach in the context of the popular Robotic Operating System and investigated its application to three systems of increasing complexity.

References

  1. L. Baresi, C. Ghezzi, and L. Mottola. Towards fine-grained automated verification of publish-subscribe architectures. In Formal Techniques for Networked and Distributed Systems, pages 131--135. Springer-Verlag, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. L. Baresi, C. Ghezzi, and L. Mottola. On accurate automatic verification of publish-subscribe architectures. In the International Conference on Software Engineering, pages 199--208, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. L. Baresi, C. Ghezzi, and L. Mottola. Loupe: Verifying publish-subscribe architectures with a magnifying lens. IEEE Transactions on Software Engineering, 37(2):228--246, March-April 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. C. Cadar, D. Dunbar, and D. Engler. KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In the USENIX Conference on Operating Systems Design and Implementation, pages 209--224, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. R. Cardell-Oliver. Conformance test experiments for distributed real-time systems. In the ACM SIGSOFT international Symposium on Software testing and Analysis, pages 159--163, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. J. Chen, R. Hierons, and H. Ural. Conditions for resolving observability problems in distributed testing. In Formal Techniques for Networked and Distributed Systems FORTE 2004, volume 3235, pages 229--242. Springer Berlin / Heidelberg, 2004.Google ScholarGoogle Scholar
  7. J. Chen, R. M. Hierons, and H. Ural. Overcoming observability problems in distributed test architectures. Inf. Process. Lett., 98(5): 177--182, June 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. D. Engler and K. Ashcraft. RacerX: effective, static detection of race conditions and deadlocks. In the Symposium on Operating systems principles, pages 237--252, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. P. T. Eugster, P. A. Felber, R. Guerraoui, and A. M. Kermarrec. The many faces of publish/subscribe. ACM Computing Surveys, 35:114--131, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. V. Ganesh and D. L. Dill. A decision procedure for bit-vectors and arrays. In the Computer Aided Verification Conference, pages 524--536, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. W. Garage, 2012. URL http://www.willowgarage.com/pages/pr2.Google ScholarGoogle Scholar
  12. D. Geels, G. Altekar, P. Maniatis, T. Roscoe, and I. Stoica. Friday: Global comprehension for distributed replay. In the Symposium on Networked Systems Design and Implementation, page 21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. P. Godefroid, N. Klarlund, and K. Sen. DART: directed automated random testing. In the ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 213--223, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. IRobot, 2012. URL http://www.irobot.com/company.Google ScholarGoogle Scholar
  15. A. Khoumsi. A temporal approach for testing distributed systems. IEEE Transactions on Software Engineering, 28:1085--1103, November 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. C. Lattner and V. Adve. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In the International Symposium on Code Generation and Optimization, March 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. G. T. Leavens. The java modeling language(jml). URL http://sourceforge.net/apps/wordpress/fixedptc/.Google ScholarGoogle Scholar
  18. A. Michlmayr, P. Fenkam, and S. Dustdar. Architecting a testing framework for publish/subscribe applications. In the International Computer Software and Applications Conference, pages 467--474, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. A. Michlmayr, P. Fenkam, and S. Dustdar. Specification-based unit testing of publish/subscribe applications. In the IEEE International Conference Workshops on Distributed Computing Systems, pages 34--, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. Quigley, K. Conley, B. P. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng. Ros: an open-source robot operating system. In International Conference on Robotics and Automation Workshop on Open Source Software, 2009.Google ScholarGoogle Scholar
  21. P. Reynolds, C. Killian, J. L. Wiener, J. C. Mogul, M. A. Shah, and A. Vahdat. Pip: detecting the unexpected in distributed systems. In the Conference on Networked Systems Design & Implementation, pages 115--128, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. M. J. Rutherford, A. Carzaniga, and A. L. Wolf. Simulation-based test adequacy criteria for distributed systems. In the ACM SIGSOFT international Symposiumon Foundations of Software Engineering, pages 231--241, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. R. Sasnauskas, O. Landsiedel, M. H. Alizai, C. Weise, S. Kowalewski, and K. Wehrle. Kleenet: discovering insidious interaction bugs in wireless sensor networks before deployment. In International Conference on Information Processing in Sensor Networks, pages 186--196, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. R. Sasnauskas, O. S. Dustmann, B. L. Kaminski, K. Wehrle, C. Weise, and S. Kowalewski. Scalable symbolic execution of distributed sys-tems. In International Conference on Distributed Computing Systems, pages 333--342, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. S. Savage, M. Burrows, G. Nelson, P. Sobalvarro, and T. Anderson. Eraser: a dynamic data race detector for multithreaded programs. ACM Trans. Comput. Syst., 15(4):391--411, Nov. 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. K. Sen. Race directed random testing of concurrent programs. In the Conference on Programming Language Design and Implementation, pages 11--21, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. E. Sherman, M. B. Dwyer, and S. Elbaum. Saturation-based testing of concurrent programs. In the European Software Engineering conference and the ACM SIGSOFT Symposium on the Foundations of Soft-ware Engineering, pages 53--62, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. J. Simsa, R. Bryant, and G. Gibson. dBug: systematic evaluation of distributed systems. In the international conference on Systems software verification, pages 3--3, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. A. Singh, P. Maniatis, T. Roscoe, and P. Druschel. Using queries for distributed monitoring and forensics. In the ACM SIGOPS/EuroSys European Conference on Computer Systems 2006, pages 389--402, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. R. Taylor, D. Levine, and C. Kelly. Structural testing of concurrent programs. IEEE Transactions on Software Engineering, 18:206--215, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. C. Wang, M. Said, and A. Gupta. Coverage guided systematic concur-rency testing. In the International Conference on Software Engineer-ing, pages 221--230, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. J. Yang, T. Chen, M. Wu, Z. Xu, X. Liu, H. Lin, M. Yang, F. Long, L. Zhang, and L. Zhou. MODIST: transparent model checking of unmodified distributed systems. In the USENIX Symposium on Networked Systems Design and Implementation, pages 213--228, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Detecting problematic message sequences and frequencies in distributed systems

              Recommendations

              Comments

              Login options

              Check if you have access through your login credentials or your institution to get full access on this article.

              Sign in

              Full Access

              • Published in

                cover image ACM SIGPLAN Notices
                ACM SIGPLAN Notices  Volume 47, Issue 10
                OOPSLA '12
                October 2012
                1011 pages
                ISSN:0362-1340
                EISSN:1558-1160
                DOI:10.1145/2398857
                Issue’s Table of Contents
                • cover image ACM Conferences
                  OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applications
                  October 2012
                  1052 pages
                  ISBN:9781450315616
                  DOI:10.1145/2384616

                Copyright © 2012 ACM

                Publisher

                Association for Computing Machinery

                New York, NY, United States

                Publication History

                • Published: 19 October 2012

                Check for updates

                Qualifiers

                • research-article

              PDF Format

              View or Download as a PDF file.

              PDF

              eReader

              View online with eReader.

              eReader
              About Cookies On This Site

              We use cookies to ensure that we give you the best experience on our website.

              Learn more

              Got it!