skip to main content
research-article

Design and implementation trade-offs for wide-area resource discovery

Published:06 October 2008Publication History
Skip Abstract Section

Abstract

We describe the design and implementation of SWORD, a scalable resource discovery service for wide-area distributed systems. In contrast to previous systems, SWORD allows users to describe desired resources as a topology of interconnected groups with required intragroup, intergroup, and per-node characteristics, along with the utility that the application derives from specified ranges of metric values. This design gives users the flexibility to find geographically distributed resources for applications that are sensitive to both node and network characteristics, and allows the system to rank acceptable configurations based on their quality for that application.

Rather than evaluating a single implementation of SWORD, we explore a variety of architectural designs that deliver the required functionality in a scalable and highly available manner. We discuss the trade-offs of using a centralized architecture as compared to a fully decentralized design to perform wide-area resource discovery. To summarize our results, we found that a centralized architecture based on 4-node server cluster sites at network-peering facilities outperforms a decentralized DHT-based resource discovery infrastructure with respect to query latency for all but the smallest number of sites. However, although a centralized architecture shows significant promise in stable environments, we find that our decentralized implementation has acceptable performance and also benefits from the DHT's self-healing properties in more volatile environments. We evaluate the advantages and disadvantages of centralized and distributed resource discovery architectures on 1000 hosts in emulation and on approximately 200 PlanetLab nodes spread across the Internet.

References

  1. Albrecht, J., Tuttle, C., Snoeren, A. C., and Vahdat, A. 2006. PlanetLab application management using Plush. SIGOPS Oper. Syst. Rev. 40, 1, 33--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Aspnes, J., Kirsch, J., and Krishnamurthy, A. 2004. Load balancing and locality in range-queriable data structures. In Proceedings of the Annual ACM SIGOPS Symposium on Principles of Distributed Computing (PODC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Aspnes, J. and Shah, G. 2003. Skip graphs. In Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. AuYoung, A., Chun, B. N., Snoeren, A. C., and Vahdat, A. 2004. Resource allocation in federated distributed computing infrastructures. In Proceedings of the Symposium on Reliable Infrastructures for XML (OASIS).Google ScholarGoogle Scholar
  5. Awerbuch, B. and Scheidler, C. 2003. Peer-to-Peer systems for prefix search. In Proceedings of the Annual ACM SIGOPS Symposium on Principles of Distributed Computing (PODC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Balazinska, M., Balakrishnan, H., and Karger, D. 2002. INS/Twine: A scalable peer-to-peer architecture for intentional resource discovery. In Proceedings of the IEEE International Conference on Program Comprehension (ICPC).Google ScholarGoogle Scholar
  7. Bavier, A., Bowman, M., Chun, B., Culler, D., Karlin, S., Muir, S., Peterson, L., Roscoe, T., Spalink, T., and Wawrzoniak, M. 2004. Operating systems support for planetary-scale network services. In Proceedings of the ACM Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Bharambe, A., Agrawal, M., and Seshan, S. 2004. Mercury: Supporting scalable multi-attribute range queries. In Proceedings of the ACM SIGCOMM Data Communications Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chang, H., Govindan, R., Jamin, S., Shenker, S., and Willinger, W. 2002. Towards capturing representative AS-level Internet topologies. In Proceedings of the ACM Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Chawathe, Y., Ramabhadran, S., Ratnasamy, S., LaMarca, A., Shenker, S., and Hellerstein, J. 2005. A case study in building layered DHT applications. In Proceedings of the ACM SIGCOMM Data Communications Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Chen, Y., Bindel, D., Song, H., and Katz, R. 2004. An algebraic approach to practical and scalable overlay network monitoring. In Proceedings of the ACM SIGCOMM Data Communications Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Chun, B. 2008. Slicestat. http://berkeley.intel-research.net/bnc/slicestat/.Google ScholarGoogle Scholar
  13. Considine, J., Byers, J., and Mayer-Patel, K. 2003. A constraint satisfication approach to testbed embedding services. In Proceedings of the Workshop on Hot Topics in Network (HotNets).Google ScholarGoogle Scholar
  14. Crainiceanu, A., Linga, P., Gehrke, J., and Shanmugasundaram, J. 2004. Querying peer-to-peer networks using P-trees. In Proceedings of the International Workshop on Web and Databases (WebDB). Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Czajkowski, K., Fitzgerald, S., Foster, I., and Kesselman, C. 2001. Grid information services for distributed resource sharing. In Proceedings of the IEEE International Symposium on High Performance Distributed Computing (HPDC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Czajkowski, K., Foster, I., Kesselman, C., Sander, V., and Tuecke, S. 2002. SNAP: A protocol for negotiating service level agreements and coordinating resource management in distributed systems. In Proceedings of the 8th Workshop on Job Scheduling Strategies for Parallel Processing. Lecture Notes in Computer Science, vol. 2537. Springer, 153--183. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Dabek, F., Cox, R., Kaahoek, F., and Morris, R. 2004. Vivaldi: A decentralized network coordinate system. In Proceedings of the ACM SIGCOMM Data Communications Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Dabek, F., Zhao, B., Druschel, P., Kubiatowicz, J., and Stoica, I. 2003. Towards a common API for structured P2P overlays. In Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS).Google ScholarGoogle Scholar
  19. DNS 1987. Domain names-implementation and specification. http://www.ietf.org/rfc/rfc1035.txt.Google ScholarGoogle Scholar
  20. Douceur, J. R. 2002. The Sybil attack. In Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Ferguson, D., Nikolaou, C., Sairamesh, J., and Yemini, Y. 1996. Economic Models for Allocating Resources in Computer Systems. World Scientific (Scott Clearwater, Ed.).Google ScholarGoogle Scholar
  22. Foster, I. and Kesselman, C. 2003. The Grid 2. Morgan Kaufmann.Google ScholarGoogle Scholar
  23. Foster, I., Kesselman, C., and Tuecke, S. 2001. The anatomy of the grid: Enabling scalable virtual organizations. Int. J. High Perform. Comput. Appl. 15, 3, 200--222. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Fu, Y., Chase, J., Chun, B., Schwab, S., and Vahdat, A. 2003. SHARP: An architecture for secure resource peering. In Proceedings of the SIGOPS Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Gupta, A., Agrawal, D., and Abbad, A. E. 2003. Approximate range selection queries in peer-to-peer systems. In Proceedings of the Conference on Innovative Data Systems Research (CIDR).Google ScholarGoogle Scholar
  26. Huang, A. and Steenkiste, P. 2003. Network-Sensitive service discovery. In Proceedings of the USENIX Symposium on Internet Technologies and Systems (USITS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Huebsch, R. 2004. PlaneTlab application manager. http://appmanager.berkeley.intel-research.net/.Google ScholarGoogle Scholar
  28. Huebsch, R., Hellerstein, J. M., Boon, N. L., Loo, T., Shenker, S., and Stoica, I. 2003. Querying the Internet with PIER. In Proceedings of the International Conference on Very Large Databases (VLDB). Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Ibaraki, T. and Katoh, N. 1988. Resource Allocation Problems: Algorithmic Approaches. MIT Press, Cambridge, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Jagadish, H. V. 1990. Linear clustering of objects with multiple attributes. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Jini. 1998. Jini homepage. http://java.sun.com/products/jini.Google ScholarGoogle Scholar
  32. Karger, D. and Ruhl, M. 2004. Simple efficient load balancing algorithms for peer-to-peer systems. In Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Kazaa. 2001. Kazaa homepage. http://www.kazaa.com/us/index.htm.Google ScholarGoogle Scholar
  34. Kee, Y.-S., Logothetis, D., Huang, R., Casanova, H., and Chien, A. 2005. Efficient resource description and high quality selection for virtual grids. In Proceedings of the IEEE International Symposium on Cluster Computing and the Gird (CCGrid). Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Krishnamurthy, B. and Wang, J. 2000. On network-aware clustering of Web clients. In Proceedings of the ACM SIGCOMM Data Communications Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. LDAP 1997. LDAP homepage. http://www.ietf.org/rfc/rfc2251.txt.Google ScholarGoogle Scholar
  37. Li, J., Stribling, J., Morris, R., Kaashoek, M. F., and Gil, T. M. 2005. A performance vs. cost framework for evaluating DHT design tradeoffs under churn. In Proceedings of the Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM).Google ScholarGoogle Scholar
  38. Linux VServer. 2003. VServer homepage. http://linux-vserver.org/.Google ScholarGoogle Scholar
  39. Litzkow, M., Livny, M., and Mutka, M. 1988. Condor--A hunter of idle workstations. In Proceedings of the IEEE International Conference on Distributed Computing Systems (ICDCS).Google ScholarGoogle Scholar
  40. Liu, C. and Foster, I. 2004. A constraint language approach to matchmaking. In Proceedings of the IEEE International Workshop on Research Issues in Data Engineering (RIDE). Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Liu, C., Yang, L., Foster, I., and Angulo, D. 2002. Design and evaluation of a resource selection framework. In Proceedings of the International Symposium on High Performance Distributed Computing (HPDC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Massie, M., Chun, B., and Culler, D. 2004. The Ganglia distributed monitoring system: Design, implementation, and experience. Parallel Comput. 30, 7 (Jul.).Google ScholarGoogle ScholarCross RefCross Ref
  43. Nath, S., Ke, Y., Gibbons, P. B., Karp, B., and Seshan, S. 2003. IrisNet: An architecture for enabling sensor-enriched Internet services. Tech. Rep. IRP-TR-03-04, Intel Research, Pittsburgh, Pennsylvania. June.Google ScholarGoogle Scholar
  44. Ng, T. S. E. and Zhang, H. 2002. Predicting Internet network distance with coordinates-based approaches. In Proceedings of the Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM).Google ScholarGoogle Scholar
  45. Ng, T. S. E. and Zhang, H. 2004. A network positioning system for the Internet. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Oppenheimer, D., Chun, B., Patterson, D., Snoeren, A. C., and Vahdat, A. 2006. Service placement in shared wide-area platforms. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Pai, V. 2008. CoTop: A slice-based top for PlanetLab. http://codeen.cs.princeton.edu/cotop/.Google ScholarGoogle Scholar
  48. Pai, V. S., Wang, L., Park, K., Pang, R., and Peterson, L. 2003. The dark side of the Web: An open proxy's view. In Proceedings of the Workshop on Hot Topics in Networks (HotNets).Google ScholarGoogle Scholar
  49. Ramabhadran, S., Ratnasamy, S., Hellerstein, J. M., and Shenker, S. 2004. Prefix hash tree. In Proceedings of the Annual ACM SIGOPS Symposium on Principles of Distributed Computing (PODC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Raman, R., Livny, M., and Solomon, M. 1998. Matchmaking: Distributed resource management for high throughput computing. In Proceedings of the IEEE International Symposium on High Performance Distributed Computing (HPDC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Raman, R., Livny, M., and Solomon, M. 2003. Policy driven heterogeneous resource co-allocation with gangmatching. In Proceedings of the IEEE International Symposium on High Performance Distributed Computing (HPDC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Ratnasamy, S., Francis, P., Handley, M., Karp, R., and Shenker, S. 2001. A content addressable network. In Proceedings of the ACM SIGCOMM Data Communications Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Red Herring Magazine. 2004. Distributed computing: We come in peace. Red Herring Mag. (Aug.).Google ScholarGoogle Scholar
  54. Reynolds, P. and Vahdat, A. 2003. Efficient peer-to-peer keyword searching. In Proceedings of the ACM/IFIP/USENIX International Middleware Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Rhea, S., Chun, B.-G., Kubiatowicz, J., and Shenker, S. 2005. Fixing the embarrassing slowness of OpenDHT on PlanetLab. In Proceedings of the Conference on Real, Large Distributed Systems (WORLDS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Rhea, S., Geels, D., Roscoe, T., and Kubiatowicz, J. 2004. Handling churn in a DHT. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Rhea, S., Godfrey, B., Karp, B., Kubiatowicz, J., Ratnasamy, S., Shenker, S., Stoica, I., and Yu, H. 2005. OpenDHT: A public DHT service and its uses. In Proceedings of the ACM SIGCOMM Data Communications Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. SLP. 1987. SLP. http://www.ietf.org/rfc/rfc2165.txt.Google ScholarGoogle Scholar
  59. Spence, D. and Harris, T. 2003. XenoSearch: Distributed resource discovery in the XenoServer open platform. In Proceedings of the IEEE International Symposium on High Performance Distributed Computing (HPDC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., and Balakrishnan, H. 2001. Chord: A scalable peer-to-peer lookup service for Internet applications. In Proceedings of the ACM SIGCOMM Data Communications Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Tang, C. and Dwarkadas, S. 2004. Hybrid global-local indexing for efficient peer-to-peer information retrieval. In Proceedings of the ACM Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Tang, C., Xu, Z., and Mahalingam, M. 2003. pSearch: Information retrieval in structured overlays. ACM SIGCOMM Comput. Commun. Rev. 33, 1, 89--94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Vahdat, A., Yocum, K., Walsh, K., Mahadevan, P., Kostić, D., Chase, J., and Becker, D. 2002. Scalability and accuracy in a large-scale network emulator. In Proceedings of the ACM USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. van Renesse, R., Birman, K., and Vogels, W. 2003. Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining. ACM Trans. Comput. Syst. 21, 2, 164--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Wawrzoniak, M., Peterson, L., and Roscoe, T. 2003. Sophia: An information plane for networked systems. In Proceedings of the Workshop on Hot Topics in Networking (HotNets).Google ScholarGoogle Scholar
  66. White, B., Lepreau, J., Stoller, L., Ricci, R., Guruprasad, S., Newbold, M., Hibler, M., Barb, C., and Joglekar, A. 2002. An integrated experimental environment for distributed systems and networks. In Proceedings of the ACM USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Zhang, X. and Schopf, J. 2004. Performance Analysis of the Globus toolkit monitoring and discovery service, MDS2. In Proceedings of the International Workshop on Middleware Performance (MP).Google ScholarGoogle Scholar

Index Terms

  1. Design and implementation trade-offs for wide-area resource discovery

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Internet Technology
        ACM Transactions on Internet Technology  Volume 8, Issue 4
        September 2008
        216 pages
        ISSN:1533-5399
        EISSN:1557-6051
        DOI:10.1145/1391949
        Issue’s Table of Contents

        Copyright © 2008 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 6 October 2008
        • Accepted: 1 December 2006
        • Revised: 1 May 2006
        • Received: 1 November 2005
        Published in toit Volume 8, Issue 4

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!