skip to main content
research-article

Load Balancing Content-Based Publish/Subscribe Systems

Published:01 December 2010Publication History
Skip Abstract Section

Abstract

Distributed content-based publish/subscribe systems suffer from performance degradation and poor scalability caused by uneven load distributions typical in real-world applications. The reason for this shortcoming is the lack of a load balancing scheme. This article proposes a load balancing solution specifically tailored to the needs of content-based publish/subscribe systems that is distributed, dynamic, adaptive, transparent, and accommodates heterogeneity. The solution consists of three key contributions: a load balancing framework, a novel load estimation algorithm, and three offload strategies. A working prototype of our solution is built on an open-sourced content-based publish/subscribe system and evaluated on PlanetLab, a cluster testbed, and in simulations. Real-life experiment results show that the proposed load balancing solution is efficient with less than 0.2% overhead; effective in distributing and balancing load originating from a single server to all available servers in the network; and capable of preventing overloads to preserve system stability, availability, and quality of service.

References

  1. Adler, M., Ge, Z., Kurose, J. F., Towsley, D., and Zabele, S. 2001. Channelization problem in large scale data dissemination. In Proceedings of the International Conference on Network Protocols (ICNP’01). IEEE Computer Society, Los Alamitos, CA, 100--109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Aekaterinidis, I. and Triantafillou, P. 2006. PastryStrings: A comprehensive content-based publish/subscribe DHT network. In Proceedings of the IEEE International Conference on Distributed Computing Systems (ICDCS’06). IEEE Computer Society, Los Alamitos, CA, 23--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Aleksy, M., Korthaus, A., and Schader, M. 2001. Design and implementation of a flexible load balancing service for CORBA-based applications. In Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA’01). IEEE Computer Society, Los Alamitos, CA.Google ScholarGoogle Scholar
  4. Altinel, M. and Franklin, M. J. 2000. Efficient filtering of XML documents for selective dissemination of information. In Proceedings of the International Conference on Very Large Data Bases (VLDB’00). Morgan-Kaufmann Publishers Inc., San Francisco, CA, 53--64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Baldoni, R., Marchetti, C., Virgillito, A., and Vitenberg, R. 2005. Content-based publish-subscribe over structured overlay networks. In Proceedings of the IEEE International Conference on Distributed Computing Systems (ICDCS’05). IEEE Computer Society, Los Alamitos, CA, 437--446. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Baldoni, R., Beraldi, R., Quema, V., Querzoni, L., and Tucci-Piergiovanni, S. 2007. TERA: Topic-based event routing for peer-to-peer architectures. In Proceedings of the Inaugural International Conference on Distributed Event-Based Systems (DEBS’07). ACM, New York, NY, 2--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Banavar, G., Chandra, T., Mukherjee, B., Nagarajarao, J., Strom, R. E., and Sturman, D. C. 1999. An efficient multicast protocol for content-based publish-subscribe systems. In Proceedings of the IEEE International Conference on Distributed Computing Systems (ICDCS’99). IEEE Computer Society, Los Alamitos, CA, 262--272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Barth, T., Flender, G., Freisleben, B., and Thilo, F. 1999. Load distribution in a CORBA environment. In Proceedings of the International Symposium on Distributed Objects and Applications (DOA’99). IEEE Computer Society, Los Alamitos, CA, 158--166. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Berman, F. and Wolski, R. 1996. Scheduling from the perspective of the application. In Proceedings of the IEEE International Symposium on High Performance Distributed Computing (HPDC’96). IEEE Computer Society, Los Alamitos, CA, 100--111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Bharambe, A. R., Rao, S., and Seshan, S. 2002. Mercury: A scalable publish-subscribe system for internet games. In Proceedings of the Workshop on Network and System Support for Games (NetGames’02). ACM, New York, NY, 3--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Cao, F. and Singh, J. P. 2004. Efficient event routing in content-based publish-subscribe service networks. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’04). 924--940.Google ScholarGoogle Scholar
  12. Cardellini, V., Colajanni, M., and Yu, P. S. 1999a. DNS dispatching algorithms with state estimators for scalable web-server clusters. World Wide Web 2, 3, 101--113. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Cardellini, V., Colajanni, M., and Yu, P. S. 1999b. Dynamic load balancing on web-server systems. IEEE Internet Comput. 3, 3, 28--39. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Carzaniga, A., Rosenblum, D. S., and Wolf, A. L. 2001. Design and evaluation of a wide-area event notification service. ACM Trans. Comput. Syst. 19, 3, 332--383. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Casalicchio, E. and Morabito, F. 2007. Distributed subscriptions clustering with limited knowledge sharing for content-based publish/subscribe systems. In Proceedings of the IEEE International Symposium on Network Computing and Applications. 105--112.Google ScholarGoogle Scholar
  16. Castelli, S., Costa, P., and Picco, G. P. 2008. HyperCBR: Large-scale content-based routing in a multidimensional space. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’08). 1714--1722.Google ScholarGoogle Scholar
  17. Castro, M., Druschel, P., Kermarrec, A.-M., and Rowstron, A. 2002. SCRIBE: A large-scale and decentralized application-level multicast infrastructure. IEEE J. Select. Areas Comm. 20, 1489--1499. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Chand, R. and Felber, P. 2005. Semantic peer-to-peer overlays for publish/subscribe networks. In Proceedings of Euro-Par Parallel Processing. Vol. 3648. 1194--1204. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Chen, C., Jacobsen, H.-A., and Vitenberg, R. 2010. Divide and conquer algorithms for publish/subscribe overlay design. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS’10). IEEE Computer Society, Los Alamitos, CA, 622--633. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Chen, Y. and Schwan, K. 2005. Opportunistic overlays: Efficient content delivery in mobile ad hoc networks. In Proceedings of the ACM/IFIP/USENIX International Conference on Middleware (Middleware’05). Springer-Verlag, Berlin, 354--374. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Cheung, A. K. Y. 2006. Dynamic load balancing in distributed content-based publish/subscribe. M.A.Sc. dissertation. University of Toronto.Google ScholarGoogle Scholar
  22. Cheung, A. K. Y. and Jacobsen, H.-A. 2006. Dynamic load balancing in distributed content-based publish/subscribe. In Proceedings of the ACM/IFIP/USENIX International Conference on Middleware (Middleware’06). Springer-Verlag, Berlin, 141--161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Cooper, B. F., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H.-A., Puz, N., Weaver, D., and Yerneni, R. 2008. PNUTS: Yahoo!’s hosted data serving platform. Proc. VLDB Endow. 1, 2, 1277--1288. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Cugola, G., Di Nitto, E., and Fuggetta, A. 2001. The JEDI event-based infrastructure and its application to the development of the OPSS WFMS. IEEE Trans. Softw. Eng. 27, 9, 827--850. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Dias, D. M., Kish, W., Mukherjee, R., and Tewari, R. 1996. A scalable and highly available web server. In Proceedings of the IEEE International Computer Conference (COMPCON’96). IEEE Computer Society, Los Alamitos, CA, 85--92. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Fabret, F., Jacobsen, H. A., Llirbat, F., Pereira, J., Ross, K. A., and Shasha, D. 2001. Filtering algorithms and implementation for very fast publish/subscribe systems. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’01). ACM, New York, 115--126. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Fawcett, T. and Provost, F. 1999. Activity monitoring: Noticing interesting changes in behavior. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’99). ACM, New York, 53--62. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Fidler, E., Jacobsen, H. A., Li, G., and Mankovski, S. 2005. The PADRES distributed publish/subscribe system. In Proceedings of the International Conference on Feature Interactions in Telecommunications and Software Systems. 12--30.Google ScholarGoogle Scholar
  29. GS1. 2008. http://www.gs1.org/docs/gdsn/gdsn_brochure.pdf.Google ScholarGoogle Scholar
  30. Gupta, A., Sahin, O. D., Agrawal, D., and Abbadi, A. E. 2004. Meghdoot: Content-based publish/subscribe over P2P networks. In Proceedings of the ACM/IFIP/USENIX International Conference on Middleware (Middleware’04). Springer-Verlag, Berlin, 254--273. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Ho, K. S. and Leong, H. V. 2000. An extended CORBA event service with support for load balancing and fault-tolerance. In Proceedings of the International Symposium on Distributed Objects and Applications (DOA’00). IEEE Computer Society, Los Alamitos, CA, 49--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Hou, S. and Jacobsen, H.-A. 2006. Predicate-based filtering of XPath expressions. In Proceedings of the International Conference on Data Engineering (ICDE’06). IEEE Computer Society, Los Alamitos, CA, 53--59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Hu, S., Muthusamy, V., Li, G., and Jacobsen, H.-A. 2008. Distributed automatic service composition in large-scale systems. In Proceedings of the International Conference on Distributed Event-based Systems (DEBS’08). ACM, New York, 233--244. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. IBM. 2008. Service Oriented Architecture (SOA). http://www-306.ibm.com/software/solutions/soa/.Google ScholarGoogle Scholar
  35. IBM, BEA Systems, Microsoft, SAP AL, and Siebel Systems. 2007. Business process execution language for web services version 1.1. http://www.ibm.com/developerworks/library/specification/ws-bpel/.Google ScholarGoogle Scholar
  36. Jul, E., Levy, H., Hutchinson, N., and Black, A. 1988. Fine-grained mobility in the emerald system. ACM Trans. Comput. Syst. 6, 1, 109--133. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Li, G. and Jacobsen, H.-A. 2005. Composite subscriptions in content-based publish/subscribe systems. In Proceedings of the ACM/IFIP/USENIX 2005 International Conference on Middleware (Middleware’05). Springer-Verlag, Berlin, 249--269. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Li, G., Cheung, A., Hou, S., Hu, S., Muthusamy, V., Sherafat, R., Wun, A., Jacobsen, H.-A., and Manovski, S. 2007. Historic data access in publish/subscribe. In Proceedings of the Inaugural International Conference on Distributed Event-Based Systems (DEBS’07). ACM, New York, 80--84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Li, G., Hou, S., and Jacobsen, H.-A. 2008a. Routing of XML and XPath queries in data dissemination networks. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS’08). IEEE Computer Society, Los Alamitos, CA, 627--638. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Li, G., Muthusamy, V., and Jacobsen, H.-A. 2008b. Adaptive content-based routing in general overlay topologies. In Proceedings of the ACM/IFIP/USENIX International Conference on Middleware (Middleware’08). Springer-Verlag, Berlin, 1--21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Li, G., Muthusamy, V., and Jacobsen, H.-A. 2010. A distributed service-oriented architecture for business process execution. ACM Trans. Web 4, 1, 1--33. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Lindermeier, M. 2000. Load management for distributed object-oriented environments. In Proceedings of the International Symposium on Distributed Objects and Applications (DOA’00). IEEE Computer Society, Los Alamitos, CA, 59--68. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Litzkow, M. J. 1987. Remote Unix: Turning idle workstations into cycle servers. In Summer USENIX Conference Proceedings.Google ScholarGoogle Scholar
  44. Microsoft. 2008. Service Oriented Architecture. http://msdn.microsoft.com/en-us/architecture/aa948857.aspx.Google ScholarGoogle Scholar
  45. MSRG. 2008. Padres project. http://msrg.org/projects/padres/.Google ScholarGoogle Scholar
  46. Mühl, G. 2001. Generic constraints for content-based publish/subscribe. In Proceedings of the International Conference on Cooperative Information Systems (CooplS’01). Springer-Verlag, Berlin, 211--225. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Mukherjee, B., Heberlein, L. T., and Levitt, K. N. 1994. Network intrusion detection. IEEE Netw. 8, 3, 26--41.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Muthusamy, V., Petrovic, M., and Jacobsen, H.-A. 2005. Effects of routing computations in content-based routing networks with mobile data sources. In Proceedings of the Annual International Conference on Mobile Computing and Networking (MobiCom’05). ACM, New York, 103--116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Muthusamy, V., Jacobsen, H.-A., Coulthard, P., Chan, A., Waterhouse, J., and Litani, E. 2007. SLA-driven business process management in SOA. In Proceedings of the 2007 Conference of the Center for Advanced Studies on Collaborative Research (CASCON’07). ACM, New York, 264--267. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Oki, B., Pfluegl, M., Siegel, A., and Skeen, D. 1993. The information bus: An architecture for extensible distributed systems. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP’93). ACM, New York, 58--68. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Opyrchal, L., Astley, M., Auerbach, J., Banavar, G., Strom, R., and Sturman, D. 2000. Exploiting IP multicast in content-based publish-subscribe systems. In Proceedings of the ACM/IFIP International Conference on Distributed Systems Platforms (Middleware’00). Springer-Verlag, Berlin, 185--207. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Othman, O., Balasubramanian, J., and Schmidt, D. C. 2003. The design of an adaptive middleware load balancing and monitoring service. In Proceedings of the International Workshop on Self-Adaptive Software (IWSAS’03). ACM, New York, 205--213. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Pallickara, S. and Fox, G. 2003. Naradabrokering: A distributed middleware framework and architecture for enabling durable peer-to-peer grids. In Proceedings of the ACM/IFIP/USENIX International Conference on Middleware (Middleware’03). Springer-Verlag, Berlin, 41--61. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Patel, J. A., Rivière, E., Gupta, I., and Kermarrec, A.-M. 2009. Rappel: Exploiting interest and network locality to improve fairness in publish-subscribe systems. Comput. Netw. 53, 13, 2304--2320. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Pereira, J., Fabret, F., Jacobsen, H.-A., Llirbat, F., and Shasha, D. 2001. WebFilter: A high-throughput XML-based publish and subscribe system. In Proceedings of the International Conference on Very Large Data Bases (VLDB’01). Morgan Kaufmann, San Francisco, CA, 723--724. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Petrovic, M., Liu, H., and Jacobsen, H.-A. 2005. G-ToPSS: Fast filtering of graph-based metadata. In Proceedings of the International Conference on World Wide Web (WWW’05). ACM, New York, 539--547. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Pietzuch, P. R. and Bacon, J. 2002. Hermes: A distributed event-based middleware architecture. In Proceedings of the International Conference on Distributed Computing Systems (ICDCSW’02). IEEE Computer Society, Los Alamitos, CA, 611--618. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Reumann, J. 2009. Pub/sub at Google. CANOE and EuroSys Summer School.Google ScholarGoogle Scholar
  59. Riabov, A., Liu, Z., Wolf, J. L., Yu, P. S., and Zhang, L. 2002. Clustering algorithms for content-based publication-subscription systems. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS’02). IEEE Computer Society, Los Alamitos, CA, 133--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Riabov, A., Liu, Z., Wolf, J. L., Yu, P. S., and Zhang, L. 2003. New algorithms for content-based publication-subscription systems. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS’03). IEEE Computer Society, Los Alamitos, CA, 678--686. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Rose, I., Murty, R., Pietzuch, P., Ledlie, J., Roussopoulos, M., and Welsh, M. 2007. Cobra: Content-based filtering and aggregation of blogs and RSS feeds. In Proceedings of the USENIX Symposium on Networked Systems Design & Implementation (NSDI’’07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Sachs, K., Kounev, S., Bacon, J., and Buchmann, A. 2009. Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark. Perform. Eval. 66, 8, 410--434. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Schuler, C., Schuldt, H., and Schek, H.-J. 2001. Supporting reliable transactional business processes by publish/subscribe techniques. In Proceedings of the 2nd International Workshop on Technologies for E-Services (TES’01). Springer-Verlag, Berlin, 118--131. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Segall, B. and Arnold, D. 1997. Elvin has left the building: A publish/subscribe notification service with quenching. In Proceedings of the Australian UNIX and Open Systems User Group Conference (AUUG’97).Google ScholarGoogle Scholar
  65. Shirriff, K. 1997. Building distributed process management on an object-oriented framework. In Proceedings of the Annual Conference on USENIX Annual Technical Conference (ATEC’97). USENIX Association, Berkeley, CA, 9--22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Tam, D., Azimi, R., and Jacobsen, H.-A. 2003. Building content-based publish/subscribe systems with distributed hash tables. In Proceedings of the International Workshop on Databases, Information Systems and Peer-to-Peer Computing. 138--152.Google ScholarGoogle Scholar
  67. Tatbul, N., Çetintemel, U., and Zdonik, S. 2007. Staying FIT: Efficient load shedding techniques for distributed stream processing. In Proceedings of the International Conference on Very Large Data Bases (VLDB’07). VLDB Endowment, 159--170. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Terpstra, W. W., Behnel, S., Fiege, L., Zeidler, A., and Buchmann, A. P. 2003. A peer-to-peer approach to content-based publish/subscribe. In Proceedings of the International Workshop on Distributed Event-Based Systems (DEBS’03). ACM, New York, 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Tibco. 2001. TIBCO software chosen as infrastructure for NASDAQ’s supermontage. www.tibco.com.Google ScholarGoogle Scholar
  70. Triantafillou, P. and Aekaterinidis, I. 2004. Content-based publish/subscribe systems over structured P2P networks. In Proceedings of the International Workshop on Distributed Event Based Systems (DEBS’04).Google ScholarGoogle Scholar
  71. Vitria. 2008. http://www.vitria.com/.Google ScholarGoogle Scholar
  72. Voulgaris, S., Rivire, E., Kermarrec, A.-M., and Steen, M. V. 2006. Sub-2-Sub: Self-organizing content-based publish subscribe for dynamic large scale collaborative networks. In Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS’06).Google ScholarGoogle Scholar
  73. Wong, T., Katz, R. H., and McCanne, S. 2000. An evaluation of preference clustering in large-scale multicast applications. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’00). 451--460.Google ScholarGoogle ScholarCross RefCross Ref
  74. Yan, W., Hu, S., Muthusamy, V., Jacobsen, H.-A., and Zha, L. 2009. Efficient event-based resource discovery. In Proceedings of the ACM International Conference on Distributed Event-Based Systems (DEBS’09). ACM, New York, 1--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Zajcew, R., Roy, P., Black, D., Peak, C., Guedes, P., Kemp, B., LoVerso, J., Leibensperger, M., Barnett, M., Rabii, F., and Netterwala, D. 1993. An OSF/1 UNIX for massively parallel multicomputers. In Proceedings of the Winter USENIX Conference. 37--55.Google ScholarGoogle Scholar
  76. Zhang, C., Krishnamurthy, A., Wang, R. Y., and Singh, J. P. 2005. Combining flexibility and scalability in a peer-to-peer publish/subscribe system. In Proceedings of the ACM/IFIP/USENIX 2005 International Conference on Middleware (Middleware’05). Springer-Verlag, Berlin, 102--123. Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Zhu, Y. and Hu, Y. 2007. Ferry: A P2P-based architecture for content-based publish/subscribe services. IEEE Trans. Parall. Distrib. Syst. 18, 5, 672--685. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Load Balancing Content-Based Publish/Subscribe Systems

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!