XL peer-to-peer pub/sub systems

Abstract

Increasingly, one of the most prominent ways to disseminate information on the Web is through “notifications” (also known as alerts), and as such they are at the core of many large-scale applications. For instance, users are notified of articles in which they are interested through RSS feeds, of posts from their friends through social networks, or of recommendation generated by various sites. Event notification usually relies on the so-called Publish-Subscribe (Pub/Sub) communication paradigm. In Pub/Sub systems, subscribers sign up for events or classes of events in order to be asynchronously notified afterward by the system. The size of such systems (with respect to events and subscriptions) keeps growing, and providing scalable implementations of Pub/Sub systems is extremely challenging. Although there exist popular examples of centralized Pub/Sub systems that currently support a large number of subscribers, such as online social networks, they periodically face formidable challenges due to peak loads and do not always offer a support for fine-grain subscriptions. In fact, providing scalability along with expressiveness in the subscription patterns calls for distributed implementations of Pub/Sub systems. In parallel, peer-to-peer (P2P) overlay networks have emerged, providing a sound and highly scalable network foundation upon which to build distributed applications including Pub/Sub systems.

In this article, we focus on fully decentralized (P2P), highly scalable, Pub/Sub systems. More specifically, we investigate how Pub/Sub and P2P research can be integrated. We define the design space and explore it in a systematic way. We expose an understanding of available design choices; provide a comprehensive classification and understanding of prominent P2P Pub/Sub systems, positioning them against the design dimensions; and highlight correlations between and implications, benefits, and shortcomings of design alternatives.

References

1. Aekaterinidis, I. and Triantafillou, P. 2005. Internet scale string attribute publish/subscribe data networks. In Proceedings of the ACM Conference on Information and Knowledge Management (CIKM’05). ACM, New York, 44--51.
2. Aekaterinidis, I. and Triantafillou, P. 2006. PastryStrings: A comprehensive content-based publish/subscribe DHT network. In Proceedings of the IEEE International Conference on Distributed Computing and Systems (ICDCS’06). IEEE, Los Alamitos, CA, 23--23.
3. Aguilera, M., Strom, R., Sturman, D., Astley, M., and Chandra, T. 1999. Matching events in a content-based subscription system. In Proceedings of the ACM Symposium on Principles of Distributed Computing (PODC’99). ACM, New York, 53--61.
4. Anceaume, E., Gradinariu, M., Datta, A., Simon, G., and Virgillito, A. 2006. A semantic overlay for self- peer-to-peer publish/subscribe. In Proceedings of the IEEE International Conference on Distributed Computing Systems (ICDCS’06). IEEE, Washington, DC, 22.
5. ApacheMQ. 2013. ActiveMQ. Retrieved November 11, 2013 from http://activemq.apache.org/.
6. Baldoni, R., Beraldi, R., Quéma, V., Querzoni, L., and Tucci-Piergiovanni, S. 2007. TERA: Topic-based event routing for peer-to-peer architectures. In Proceedings of the International Conference on Distributed Event-Based Systems (DEBS’07). ACM, New York, 2--13.
7. Baldoni, R., Marchetti, C., Virgillito, A., and Vitenberg, R. 2005. Content-Based Publish-Subscribe over Structured Overlay Networks. In IEEE International Conference on Distributed Computing Systems (ICDCS). IEEE, Los Alamitos, CA, 437--446.
8. Banavar, G., Tushar, C., Mukherjee, B., Nagarajarao, J., Strom, R., and Sturman, D. 1999. An efficient multicast protocol for content-based publish-subscribe systems. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS’99). IEEE, Los Alamitos, CA, 262.
9. Banerjee, S., Lee, S., Bhattacharjee, B., and Srinivasan, A. 2006. Resilient multicast using overlays. IEEE/ACM Trans. Networking 14, 2, 237--248.
10. Beaumont, O., Kermarrec, A.-M., Marchal, L., and Riviere, E. 2007. VoroNet: A scalable object network based on Voronoi tessellations. In Proceedings of the International Conference on Parallel and Distributed Systems (IPDPS’07). IEEE, Los Alamitos, CA, 1--10.
11. Bender, M., Michel, S., Parkitny, S., and Weikum, G. 2006. A comparative study of pub/sub methods in structured P2P networks. In Proceedings of the International Conference on Databases, Information Systems, and Peer-to-Peer Computing (DBISP2P). Springer-Verlag, Berlin, 385--396.
12. Bharambe, A., Agrawal, M., and Seshan, S. 2004. Mercury: Supporting scalable multi-attribute range queries. In Proceedings of the ACM SIGCOMM Conference on Data Communication. ACM, New York, 353--366.
13. Bharambe, A., Douceur, J. R., Lorch, J. R., Moscibroda, T., Pang, J., Seshan, S., and Zhuang, X. 2008. Donnybrook: Enabling large-scale, high-speed, peer-to-peer games. In Proceedings of the ACM SIGCOMM Conference on Data Communication. ACM, New York, 389--400.
14. Bharambe, A., Pang, J., and Seshan, S. 2006. Colyseus: A distributed architecture for online multiplayer games. In Proceedings of the Conference on Networked Systems Design & Implementation (NSDI’’06). USENIX, Berkeley, CA, 12.
15. Bianchi, S., Felber, P., and Gradinariu, M. 2007. Content-based publish/subscribe using distributed R-Trees. In Euro-Par 2007 Parallel Processing. Lecture Notes in Computer Science, Springer, Berlin, 537--548.
16. Boulanger, J.-S., Kienzle, J., and Verbrugge, C. 2006. Comparing interest management algorithms for massively multiplayer games. In Proceedings of the ACM SIGCOMM Workshop on Network and System Support for Games (NetGames'06). ACM, New York.
17. Carzaniga, A., Di Nitto, E., Rosenblum, D., and Wolf, A. 1998. Issues in supporting event-based architectural styles. In Proceedings of the International Software Architecture Workshop. ACM, New York, 17--20.
18. Carzaniga, A., Rosenblum, D. S., and Wolf, Al. L. 2001. Design and evaluation of a wide-area event notification service. ACM Trans. Comput. Syst. 19, 3, 332--383.
19. Cassandra. 2013. Welcome to Cassandra. Retrieved November 11, 2013 from http://cassandra.apache.org/.
20. Castro, M., Druschel, P., Kermarrec, A.-M., and Rowstron, A. 2002. SCRIBE: A large-scale and decentralized application-level multicast infrastructure. IEEE J. Sel. Areas Commun. 20, 8, 1489--1499.
21. Castro, M., Jones, M., Kermarrec, A.-M., Rowstron, A., Theimer, M., Wang, H., and Wolman, A. 2003. An evaluation of scalable application-level multicast using peer-to-peer overlay networks. In IEEE INFOCOM'03. IEEE, San Francisco, CA, USA, 1510--1520.
22. Chen, J., DeWitt, D. J., Tian, F., and Wang, Y. 2000. NiagaraCQ: A scalable continuous query system for internet databases. In ACM SIGMOD International Conference on Management of Data. ACM, Dallas, Texas, USA, 379--390.
23. Chockler, G., Melamed, R., Tock, Y., and Vitenberg, R. 2007. SpiderCast: A scalable interest-aware overlay for topic-based pub/sub communication. In Proceedings of the International Conference on Distributed Event-Based Systems (DEBS’07). ACM, New York, 14--25.
24. Cugola, G., Di Nitto, E., and Fuggetta, A. 2001. The JEDI event-based infrastructure and its application to the development of the OPSS WFMS. IEEE Trans. Software Eng. 27, 9, 827--850.
25. Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D., and Terry, D. 1987. Epidemic algorithms for replicated database maintenance. In Proceedings of the ACM Symposium on Principles of Distributed Computing (PODC’87). ACM, New York, 1--12.
26. Eugster, P., Felber, P. A., Guerraoui, R., and Kermarrec, A.-M. 2003a. The many faces of publish/subscribe. Comput. Surv. 35, 2, 114--131.
27. Eugster, P., Guerraoui, R., Handurukande, S., Kouznetsov, P., and Kermarrec, A.-M. 2003b. Lightweight probabilistic broadcast. ACM Trans. Comput. Syst. 21, 4, 341--374.
28. Fabret, F., Jacobsen, A., Llirbat, F., Pereira, J., Ross, K., and Shasha, D. 2001. Filtering algorithms and implementation for very fast publish/subscribe systems. In Proceedings of the ACM SIGCOMM Conference on Data Communication. ACM, New York, 115--126.
29. Ganesh, A. J., Kermarrec, A.-M., and Massoulié, L. 2003. Peer-to-peer membership management for gossip-based protocols. IEEE Trans. Comput. 52, 2, 139--149.
30. Gao, J. and Steenkiste, P. 2004. An adaptive protocol for efficient support of range queries in DHT-based systems. In Proceedings of the IEEE International Conference on Network Protocols (ICNP’04). IEEE, 239--250.
31. Garrod, C., Manjhi, A., Ailamaki, A., Maggs, B. M., Mowry, T. C., Olston, C., and Tomasic, A. 2008. Scalable query result caching for web applications. Proc. VLDB Endowment 1, 1, 550--561.
32. Gnutella. 2013. Wego. Retrieved November 11, 2013 from http://gnutella.wego.com/.
33. Gryphon. 2005. The Gryphon Project. Retreived November 11, 2013 from http://www.research.ibm.com/distributedmessaging/gryphon.html.
34. Gupta, A., Sahin, O. D., Agrawal, D., and El Abbadi, A. 2004. Meghdoot: Content-based publish/subscribe over P2P networks. In Proceedings of the ACM/IFIP/USENIX International Middleware Conference (Middleware’04). Springer Verlag, Berlin, 254--273.
35. Gupta, I., Kermarrec, A.-M., and Ganesh, A. 2006. Efficient and adaptive epidemic-style protocols for reliable and scalable multicast. IEEE Trans. Parallel Distrib. Syst. 17, 7, 593--605.
36. Hanson, E., Carnes, C., Huang, L., Konyala, M., Noronha, L., Parthasarathy, S., Park, J. B., and Vernon, A. 1999. Scalable trigger processing. In Proceedings of the International Conference on Data Engineering (ICDE’99). IEEE, New York, 266.
37. Hua Chu, Y., Rao, S., and Zhang, H. 2002. A case for end system multicast. IEEE J. Selected Areas in Communications (JSAC) 20, 8.
38. IBM News. 2013. History of IBM: 2000s. Retrieved November 11, 2013 from http://www-03.ibm.com/ibm/history/history/year_2000.html.
39. Jelasity, M., Voulgaris, S., Guerraoui, R., Kermarrec, A.-M., and van Steen, M. 2007. Gossip-based peer sampling. ACM Trans. Comput. Syst. 25, 3, 8.
40. JMS Benchmark. 2008. JMS Performance Benchmarks. Retrieved November 11, 2013 from http://www.codeproject.com/Articles/26461/JMS-Performance-Benchmarks.
41. Kafka. 2013. Kafka 0.8 Documentation. Retrieved November 11, 2013 from http://kafka.apache.org/design.html.
42. Karp, B., Ratnasamy, S., Rhea, S., and Shenker, S. 2004. Spurring adoption of DHTs with OpenHash, a public DHT service. In Proceedings of the 3rd International Workshop on Peer-to-Peer Systems (IPTPS’04). Springer, Berlin, 195--205.
43. Kazaa. 2006. http://www.kazaa.com/.
44. Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., Wells, C., and Zhao, B. 2000. OceanStore: An architecture for global-scale persistent storage. SIGPLAN Not. 35, 11, 190--201.
45. Lehn, M., Leng, C., Rehner, R., Triebel, T., and Buchmann, A. 2011. An online gaming testbed for peer-to-peer architectures. In Proceedings of the ACM SIGCOMM Conference on Data Communication. ACM, New York, 474--475.
46. LiveJournal. 2013. Homepage. Retrieved November 11, 2013 from http://www.livejournal.com.
47. Melamed, R. and Keidar, I. 2008. Araneola: A scalable reliable multicast system for dynamic environments. J. Parallel Distrib. Comput. 68, 12, 1539--1560.
48. Object-Management Group. 1998. CORBAservices: Common Object Services Specification.
49. Object-Management Group. 2000. CORBAservices Notification Service Specification. Retrieved November 11, 2013 from http://www.omg.org/.
50. Oki, B., Pfluegl, M., Siegel, A., and Skeen, D. 1993. The Information Bus: An architecture for extensible distributed systems. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP’93). ACM, New York, 58--68.
51. Patel, J., Rivière, É, Gupta, I., and Kermarrec, A.-M. 2009. Rappel: Exploiting interest and network locality to improve fairness in publish-subscribe systems. Comput. Networks 53, 13, 2304--2320.
52. Pietzuch, P. and Bacon, J. 2002. Hermes: A distributed event-based middleware architecture. In Proceedings of the Workshop on Distributed Event Based Systems (DEBS’02). ACM, New York, 611--618.
53. Pitoura, T., Ntarmos, N., and Triantafillou, P. 2006. Replication, Load Balancing and Efficient Range Query Processing in DHTs. In International conference on Advances in Database Technology (EDBT06). Springer-Verlag, Munich, Germany, 131--148.
54. Plaxton, G., Rajaraman, R., and Richa, A. 1997. Accessing Nearby Copies of Replicated Objects in a Distributed Environment. In ACM Symposium on Parallel Algorithms and Architectures (SPAA). ACM, Newport, Rhode Island, USA, 311--320.
55. RabbitMQ. http://www.rabbitmq.comcom/. (????).
56. Ramabhadran, S., Ratnasamy, S., Hellerstein, J. M., and Shenker, S. 2004. Prefix Hash Tree - An Indexing Data Structure over Distributed Hash Tables. Technical Report. IRB Tech Repor.
57. Ramasubramanian, V., Peterson, R., and Gün Sirer, E. 2006. Corona: A high performance publish-subscribe system for the World Wide Web. In Proceedings of the Conference on Networked Systems Design & Implementation (NSDI’’06). USENIX, Berkeley, CA, 2--2.
58. Ratnasamy, S., Francis, P., Handley, M., Karp, R. M., and Shenker, S. 2001. A scalable content-addressable network. In Proceedings of the ACM SIGCOMM Conference on Data Communication. ACM, New York, 161--172.
59. Rose, I., Murty, R., Pietzuch, P., Ledlie, J., Roussopoulos, M., and Welsh, M. 2007. Cobra: Content based filtering and aggregation of blogs and RSS feeds. In Proceedings of the USENIX Conference on Networked Systems Design and Implementation. USENIX, Berkeley, CA, 3.
60. Rowstron, A. and Druschel, P. 2001a. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms (Middleware’01). Springer, Berlin, 329--350.
61. Rowstron, A. and Druschel, P. 2001b. Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP’01). ACM, New York, 188--201.
62. Sandler, D., Mislove, A., Post, A., and Druschel, P. 2005. FeedTree: Sharing web micronews with peer-to-peer event notification. In Proceedings of the International Conference on Peer-to-Peer Systems (IPTPS’05). Springer-Verlag, Berlin, 141--151.
63. Skype. Homepage. Retrieved November 11, 2013 from http://www.skype.com/.
64. Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., and Balakrishnan, H. 2001. Chord: A scalable peer-to-peer lookup service for internet applications. In Proceedings of the ACM SIGCOMM Conference on Data Communication. ACM, New York, 149--160.
65. Sun, X. 2007. SCAN: A small-world structured P2P overlay for multi-dimensional queries. In Proceedings of the International Conference on World Wide Web (WWW’07). ACM, BNew York, 1191--1192.
66. Sun Microsystems, Inc. 2000. Jini#8482; Technology Core Platform Specification. Retreived November 11, 2013 from http://www-csag.ucsd.edu/teaching/cse291s03/Readings/core1_2.pdf.
67. Terpstra, W., Behnel, S., Fiege, L., Zeidler, A., and Buchmann, A. 2003. A peer-to-peer approach to content-based publish/subscribe. In Proceedings of the International Conference on Distributed Event-Based Systems (DEBS’03). ACM, New York, 1--8.
68. Tibco. 2013. Homepage. Retrieved November 11, 2013 from http://www.tibco.com/.
69. Triantafillou, P. and Aekaterinidis, I. 2004. Publish-subscribe over structured P2P networks. In Proceedings of the International Conference on Distributed Event-Based Systems (DEBS’04). ACM, New York.
70. Triantafillou, P. and Pitoura, T. 2003. Towards a unifying framework for complex query processing over structured peer-to-peer data networks. In Proceedings of the Workshop on Databases, Information Systems, and Peer-to-Peer Computing (VLDB’03). Springer, Berlin.
71. Twitter. 2013. Welcome to Twitter. Retrieved November 11, 2013 from http://www.twitter.com/.
72. Vitria. 2013. Homepage. Retrieved November 11, 2013 from http://www.vitria.com/.
73. Voulgaris, S., Gavidia, D., and van Steen, M. 2005. CYCLON: Inexpensive membership management for unstructured P2P overlays. J. Netw. Syst. Manage. 13, 2.
74. Voulgaris, S., Rivière, E., Kermarrec, A.-M., and van Steen, M. 2006. Sub-2-Sub: Self-organizing content-based publish and subscribe for dynamic and large scale collaborative networks. In Proceedings of the 5th International Workshop on Peer-to-Peer Systems (IPTPS’06). USENIX, Berkeley, CA.
75. Yeo, C. K., Leea, B. S., and Er., M. H. 2004. A survey of application level multicast techniques. Comput. Commun. 27, 15, 1547--1568.
76. Zhang, C., Krishnamurthy, A., Wang, R. Y., and Singh, J. P. 2005. Combining flexibility and scalability in a peer-to-peer publish/subscribe system. In Proceedings of the ACM/IFIP/UseNix 6th International Middleware Conference (Middleware’05). Springer-Verlag, Berlin, 102--123.
77. Zhao, B., Joseph, A., and Kubiatowicz, J. 2001. Tapestry: An Infrastructure for Fault-Tolerant Wide-Area Location and Routing. Tech. rep. UCB/CSD-01-1141. University of California, Berkley.
78. Zhuang, S. Q., Zhao, B. Y., Joseph, A. D., Katz, R. H., and Kubiatowicz, J. D. 2001. Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination. In Proceedings of the International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV). ACM, New York, 11--20.

Index Terms

• Information systems

  • Data management systems

    • Database management system engines

      • Parallel and distributed DBMSs

• Software and its engineering

  • Software creation and management

    • Designing software

  • Software organization and properties

    • Software system structures

      • Distributed systems organizing principles