Abstract
Many large-scale utility computing infrastructures comprise heterogeneous hardware and software resources. This raises the need for scalable resource selection services that identify resources that match application requirements. Such a service must provide an efficient lookup in spite of changing resource attributes such as disk size, changing application requirements such as installed software libraries, and changing system composition as resources join or leave. We present a fully decentralized, self-managing Resource Selection Service (RSS) algorithm by which resources autonomously select themselves when their attributes match a query. An application specifies what it expects from a resource by means of a conjunction of (attribute,value-range) pairs, which are matched against the attribute values of resources. The set of search attributes can also be updated online to reflect new requirements. We show that our solution scales in the number of resources and in the number of attributes, while being relatively insensitive to churn and other membership changes like node failures. Our RSS continuously self-adapts its routing structure in response to variations in the distribution of node attributes and queries. We show that this autonomous optimization maintains performance and availability in a long-lived service even when the set of application requirements used to select resources changes.
- Aberer, K., Datta, A., and Hauswirth, M. 2005. Multifaceted simultaneous load balancing in DHT-based P2P systems: A new game with old balls and bins. In Self-Star Properties in Complex Information Systems. Lecture Notes in Computer Science, vol. 3460. Springer, 373--391. Google Scholar
Digital Library
- Albrecht, J., Oppenheimer, D., Vahdat, A., and Patterson, D. A. 2008. Design and implementation trade-offs for wide-area resource discovery. ACM Trans. Interet Technol. 8, 4. Google Scholar
Digital Library
- Anderson, D. P. and Reed, K. 2009. Celebrating diversity in volunteer computing. In Proceedings of the Hawaii International Conference on System Sciences. Google Scholar
Digital Library
- Anderson, T. and Roscoe, T. 2006. Learning from PlanetLab. In Proceedings of the Workshop on Real, Large Distributed Systems. Google Scholar
Digital Library
- Beaumont, O., Kermarrec, A.-M., Marchal, L., and Rivière, E. 2007. VoroNet: A scalable object network based on Voronoi tessellations. In Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS).Google Scholar
- Bharambe, A. R., Agrawal, M., and Seshan, S. 2004. Mercury: Supporting scalable multi-attribute range queries. In Proceedings of ACM SIGCOMM. 353--366. Google Scholar
Digital Library
- Cai, M., Frank, M., Chen, J., and Szekely, P. 2003. MAAN: A multi-attribute addressable network for grid information services. In Proceedings of the International Workshop on Grid Computing. Google Scholar
Digital Library
- Coppola, M., Jégou, Y., Matthews, B., Morin, C., Prieto, L. P., Sánchez, O. D., Yang, E., and Yu, H. 2008. Virtual organization support within a grid-wide operating system. IEEE Internet Comput. 12, 2. Google Scholar
Digital Library
- Costa, P., Napper, J., Pierre, G., and van Steen, M. 2009. Autonomous resource selection for decentralized utility computing. In Proceedings of the IEEE International Conference on Distributed Computing Systems (ICDCS). Google Scholar
Digital Library
- DAS-3. http://www.cs.vu.nl/das3/.Google Scholar
- Drost, N., van Nieuwpoort, R. V., and Bal, H. 2006. Simple locality-aware co-allocation in peer-to-peer supercomputing. In Proceedings of the Workshop on Global and Peer-2-Peer Computing.Google Scholar
- Fiscato, M., Costa, P., and Pierre, G. 2008. On the feasibility of decentralized grid scheduling. In Proceedings of the International Workshop on Decentralized Self Management for Grids, P2P, and User Communities (Selfman’08). Google Scholar
Digital Library
- Foster, I. and Iamnitchi, A. 2003. On death, taxes, and the convergence of peer-to-peer and grid computing. In Proceedings of the International Workshop on Peer-to-Peer Systems.Google Scholar
- Ganesan, P., Yang, B., and Garcia-Molina, H. 2004. One torus to rule them all: Multi-dimensional queries in P2P systems. In Proceedings of the International Workshop on the Web and Databases. Google Scholar
Digital Library
- Gupta, A., Sahin, O. D., Agrawal, D., and Abbadi, A. E. 2004. Meghdoot: Content-based publish/subscribe over P2P networks. In Proceedings of the International Middleware Conference. Google Scholar
Digital Library
- Iosup, A., Jan, M., Sonmez, O., and Epema, D. H. J. 2007. On the dynamic resource availability in grids. In Proceedings of the International Conference on Grid Computing. Google Scholar
Digital Library
- Iosup, A., Li, H., Jan, M., Anoep, S., Dumitrescu, C., Wolters, L., and Epema, D. H. J. 2008. The grid workloads archive. Future Gen. Comput. Syst. 24, 7, 672--686. Google Scholar
Digital Library
- Jelasity, M. and Kermarrec, A.-M. 2006. Ordered slicing of very large-scale overlay networks. In Proceedings of P2P. Google Scholar
Digital Library
- Jelasity, M., Voulgaris, S., Guerraoui, R., Kermarrec, A.-M., and van Steen, M. 2007. Gossip-based peer sampling. ACM Trans. Comput. Syst. 25, 3. Google Scholar
Digital Library
- Jelasity, M., Montresor, A., Jesi, G. P., and Voulgaris, S. The Peersim simulator. http://peersim.sf.net.Google Scholar
- Ranjan, R., Harwood, A., and Buyya, R. 2008. Peer-to-peer based resource discovery in global grids: A tutorial. IEEE Comm. Surveys Tutorials 10, 2. Google Scholar
Digital Library
- Ratnasamy, S., et al. 2001. A scalable content addressable network. In Proceedings of SIGCOMM. 161--172. Google Scholar
Digital Library
- Rhea, S., Geels, D., Roscoe, T., and Kubiatowicz, J. 2004. Handling churn in a DHT. In Proceedings of the USENIX Technical Conference. Google Scholar
Digital Library
- Sacha, J., Napper, J., Stratan, C., and Pierre, G. 2010. Adam2: Reliable distribution estimation in decentralised environments. In Proceedings of the IEEE International Conference on Distributed Computing Systems (ICDCS). Google Scholar
Digital Library
- Sanchez-Artigas, M., Garcia-Lopez, P., and Skarmeta, A. F. G. 2008. On the feasibility of dynamic superpeer ratio maintenance. In Proceedings of the Conference on P2P, 333--342. Google Scholar
Digital Library
- Saroiu, S., Gummadi, K. P., and Gribble, S. D. 2003. Measuring and analyzing the characteristics of Napster and Gnutella hosts. Multimed. Syst. 9, 2, 170--184. Google Scholar
Digital Library
- Schmidt, C. and Parashar, M. 2003. Flexible information discovery in decentralized distributed systems. In Proceedings of the 12th International Symposium on High-Performance Distributed Computing (HPDC). Google Scholar
Digital Library
- Scholtes, I., Botev, J., Hohfeld, A., Schloss, H., and Esch, M. 2008. Awareness-driven phase transitions in very large scale distributed systems. In Proceedings of the 2nd IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO). 25--34. Google Scholar
Digital Library
- Schütt, T., et al. 2007. A structured overlay for multi-dimensional range queries. In Proceedings of the International Euro-Par Conference. Google Scholar
Digital Library
- Sharma, B., Chudnovsky, V., Hellerstein, J. L., Rifaat, R., and Das, C. R. 2011. Modeling and synthesizing task placement constraints in google compute clusters. In Proceedings of the 2nd ACM Symposium on Cloud Computing (SOCC). Google Scholar
Digital Library
- Snyder, P. L., Greenstadt, R., and Valetto, G. 2009. Myconet: A fungi-inspired model for superpeer-based peer-to-peer overlay topologies. In Proceedings of the 3rd IEEE International Conference on Self-Adaptive and Organizing Systems (SASO). 40--50. Google Scholar
Digital Library
- Spence, D. and Harris, T. 2003. Distributed resource discovery in the XenoServer open platform. In Proceedings of the 12th International Symposium on High-Performance Distributed Computing (HPDC). Google Scholar
Digital Library
- Steele, T., Vishnumurthy, V., and Francis, P. 2008. A parameter-free load balancing mechanism for p2p networks. In Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS). Google Scholar
Digital Library
- Tanin, E., Harwood, A., and Samet, H. 2007. Using a distributed quadtree in peer-to-peer networks. VLDB J. Google Scholar
Digital Library
- van Renesse, R., Birman, K. P., and Vogels, W. 2003. Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining. ACM Trans. Comput. Syst. 21, 2, 164--206. Google Scholar
Digital Library
- van Steen, M. and Ballintijn, G. 2002. Achieving scalability in hierarchical location services. In Proceedings of the International Computer Software and Applications Conference. Google Scholar
Digital Library
- Voulgaris, S. and van Steen, M. 2005. Epidemic-style management of semantic overlays for content-based searching. In Proceedings of the International Euro-Par Conference on Parallel Processing. Google Scholar
Digital Library
- XtremLab Project. http://xw01.lri.fr:4320/.Google Scholar
- Yalagandula, P. and Dahlin, M. 2004. A scalable distributed information management system. In Proceedings of the ACM SIGCOMM. 379--390. Google Scholar
Digital Library
- Zanikolas, S. and Sakellariou, R. 2005. A taxonomy of grid monitoring systems. Future Gen. Comput. Syst. Google Scholar
Digital Library
Index Terms
The XtreemOS Resource Selection Service
Recommendations
Autonomous Resource Selection for Decentralized Utility Computing
ICDCS '09: Proceedings of the 2009 29th IEEE International Conference on Distributed Computing SystemsMany large-scale utility computing infrastructures comprise heterogeneous hardware and software resources. This raises the need for scalable resource selection services, which identify resources that match application requirements, and can potentially ...
Using Frequent Workload Patterns in Resource Selection for Grid Jobs
APSCC '08: Proceedings of the 2008 IEEE Asia-Pacific Services Computing ConferenceResource selection is an important issue of grid computing. If a grid job can stably gain enough CPU time from the same resources, not only the execution time of the job but also the frequency of resource reallocation is effectively minimized. However, ...
Dynamic resource selection heuristics for a non-reserved bidding-based Grid environment
A Grid system is comprised of large sets of heterogeneous and geographically distributed resources that are aggregated as a virtual computing platform for executing large-scale scientific applications. As the number of resources in Grids increases ...








Comments