Abstract
This article focuses on the multireceiver data dissemination problem. Initially, IP multicast formed the basis for efficiently supporting such distribution. More recently, overlay networks have emerged to support point-to-multipoint communication. Both techniques focus on constructing trees rooted at the source to distribute content among all interested receivers. We argue, however, that trees have two fundamental limitations for data dissemination. First, since all data comes from a single parent, participants must often continuously probe in search of a parent with an acceptable level of bandwidth. Second, due to packet losses and failures, available bandwidth is monotonically decreasing down the tree.
To address these limitations, we present Bullet, a data dissemination mesh that takes advantage of the computational and storage capabilities of end hosts to create a distribution structure where a node receives data in parallel from multiple peers. For the mesh to deliver improved bandwidth and reliability, we need to solve several key problems: (i) disseminating disjoint data over the mesh, (ii) locating missing content, (iii) finding who to peer with (peering strategy), (iv) retrieving data at the right rate from all peers (flow control), and (v) recovering from failures and adapting to dynamically changing network conditions. Additionally, the system should be self-adjusting and should have few user-adjustable parameter settings. We describe our approach to addressing all of these problems in a working, deployed system across the Internet. Bullet outperforms state-of-the-art systems, including BitTorrent, by 25-70% and exhibits strong performance and reliability in a range of deployment settings. In addition, we find that, relative to tree-based solutions, Bullet reduces the need to perform expensive bandwidth probing.
- Ahlswede, R., Cai, N., Li, S.-Y. R., and Yeung, R. W. 2000. Network information flow. IEEE Trans. Inform. Theor. 46, 4 (Jul.), 1204--1216. Google Scholar
Digital Library
- Albrecht, J., Tuttle, C., Snoeren, A. C., and Vahdat, A. 2006. PlanetLab application management using Plush. ACM SIGOPS Operat. Syst. Rev. 40 1 (Jan.), 33--40. Google Scholar
Digital Library
- Andersen, D. G., Balakrishnan, H., Kaashoek, M. F., and Morris, R. 2001. Resilient overlay networks. In Proceedings of the Symposium on Operating Systems Principles. Google Scholar
Digital Library
- Banerjee, S., Bhattacharjee, B., and Kommareddy, C. 2002. Scalable application layer multicast. In Proceedings of ACM SIGCOMM. Google Scholar
Digital Library
- Bharambe, A., Herley, C., and Padmanabhan, V. 2006. Analyzing and improving a BitTorrent network's performance mechanisms. In Proceedings of IEEE INFOCOM.Google Scholar
- Bickson, D. and Malkhi, D. 2005. The Julia content distribution network. In Proceedings of the Second Workshop on Real, Large Distributed Systems (WORLDS). Google Scholar
Digital Library
- Birman, K., Hayden, M., Ozkasap, O., Xiao, Z., Budiu, M., and Minsky, Y. 1999. Bimodal multicast. ACM Trans. Comput. Syst. 17, 2 (May), 41--88. Google Scholar
Digital Library
- Bloom, B. 1970. Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13, 7 (Jul.), 422--426. Google Scholar
Digital Library
- Broder, A. 1997. On the resemblance and containment of documents. In Proceedings of the Conference on Compression and Complexity of Sequences (SEQUENCES'97). Google Scholar
Digital Library
- Byers, J. W., Considine, J., Mitzenmacher, M., and Rost, S. 2002. Informed content delivery across adaptive overlay networks. In Proceedings of ACM SIGCOMM. Google Scholar
Digital Library
- Byers, J. W., Luby, M., Mitzenmacher, M., and Rege, A. 1998. A digital fountain approach to reliable distribution of bulk data. In Proceedings of ACM SIGCOMM. Google Scholar
Digital Library
- Calvert, K., Doar, M., and Zegura, E. W. 1997. Modeling Internet topology. IEEE Commun. Mag. 35, 6 (Jun.), 160--163. Google Scholar
Digital Library
- Castro, M., Druschel, P., Kermarrec, A.-M., Nandi, A., Rowstron, A., and Singh, A. 2003. Splitstream: High-bandwidth content distribution in cooperative environments. In Proceedings of the 19th ACM Symposium on Operating System Principles. Google Scholar
Digital Library
- Chang, H., Govindan, R., Jamin, S., Shenker, S., and Willinger, W. 2002. Towards capturing representative AS-level Internet topologies. In Proceedings of ACM SIGMETRICS. Google Scholar
Digital Library
- Cherkasova, L. and Lee, J. 2003. FastReplica: Efficient large file distribution within content delivery networks. In Proceedings of the 4th USENIX Symposium on Internet Technologies and Systems. Google Scholar
Digital Library
- Chou, P. A., Wu, Y., and Jain, K. 2003. Practical network coding. In Proceedings of the Allerton Conference on Communication, Control, and Computing.Google Scholar
- Cohen, B. 2003. Incentives build robustness in bittorrent. In Proceedings of the First Workshop on the Economics of Peer-to-Peer Systems.Google Scholar
- Cohen, R. and Kaempfer, G. 2001. A unicast-based approach for streaming multicast. In Proceeding of INFOCOM. 440--448.Google Scholar
- Dabek, F., Li, J., Sit, E., Kaashoek, F., Morris, R., and Blake, C. 2004. Designing a DHT for low latency and high throughput. In Proceedings of the USENIX/ACM Symposium on Networked Systems Design and Implementation (NSDI, San Francisco, CA). Google Scholar
Digital Library
- Dalal, Y. K. and Metcalfe, R. M. 1978. Reverse path forwarding of broadcast packets. Commun. ACM 21, 12, 1040--1048. Google Scholar
Digital Library
- Deering, S. E. 1991. Multicast routing in a datagram internetwork. Ph.D. dissertation. Stanford University, Stanford, CA. Google Scholar
Digital Library
- Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D., and Terry, D. 1987. Epidemic algorithms for replicated database maintenance. In Proceedings of the Sixth Symposium on Principles of Distributed Computing. 1--12. Google Scholar
Digital Library
- Eugster, P., Guerraoui, R., Handurukande, S., Kouznetsov, P., and Kermarrec, A.-M. 2003. Lightweight probabilistic broadcast. ACM Trans. Comput. Syst. 21, 4, 341--374. Google Scholar
Digital Library
- Eugster, P., Handurukande, S., Guerraoui, R., Kermarrec, A.-M., and Kouznetsov, P. 2001. Lightweight probabilistic broadcast. In Proceedings of The International Conference on Dependable Systems and Networks (DSN). Google Scholar
Digital Library
- Floyd, S., Handley, M., Padhye, J., and Widmer, J. 2000. Equation-based congestion control for unicast applications. In Proceedings of ACM SIGCOMM (Stockholm, Sweden). 43-- 56. Google Scholar
Digital Library
- Floyd, S., Jacobson, V., Liu, C.-G., McCanne, S., and Zhang, L. 1997. A reliable multicast framework for light-weight sessions and application level framing. IEEE / ACM Trans. Network. 5, 6, 784--803. Google Scholar
Digital Library
- Gkantsidis, C., Miller, J., and Rodriguez, P. 2006. Anatomy of a p2p content distribution system with network coding. In Proceedings of the Second International Peer to Peer Symposium (IPTPS).Google Scholar
- Gkantsidis, C. and Rodriguez, P. R. 2005. Network coding for large scale content distribution. In Proceedings of IEEE INFOCOM.Google Scholar
- Goyal, V. K. 2001. Multiple description coding: Compression meets the network. IEEE Signal Process. Mag. 18, 5, 74--93.Google Scholar
Cross Ref
- Hu, N. and Steenkiste, P. 2003. Evaluation and characterization of available bandwidth probing techniques. IEEE J. Select are Commun. 21, 6, 879--895. Google Scholar
Digital Library
- Hua Chu, Y., Ganjam, A., Ng, T. S. E., Rao, S. G., Sripanidkulchai, K., Zhan, J., and Zhang, H. 2004. Early experience with an Internet broadcast system based on overlay multicast. In Proceedings of the USENIX 2004 Annual Technical Conference. Google Scholar
Digital Library
- Hua Chu, Y., Rao, S., and Zhang, H. 2000. A case for end system multicast. In Proceedings of the ACM Sigmetrics 2000 International Conference on Measurement and Modeling of Computer Systems. Google Scholar
Digital Library
- Hua Chu, Y., Rao, S. G., Seshan, S., and Zhang, H. 2001. Enabling Conferencing Applications on the Internet using an overlay multicast architecture. In Proceedings of ACM SIGCOMM. Google Scholar
Digital Library
- InformationWeek. 2004. Go online to http://informationweek.com/story/showArticle.jhtml? articleID=50900297.Google Scholar
- Jain, K., Lovasz, L., and Chou, P. A. 2005. Building scalable and robust peer-to-peer overlay networks for broadcasting using network coding. In Proceedings of the ACM Conference on Principles Of Distributed Computing (PODC). Google Scholar
Digital Library
- Jain, K., Mahdian, M., and Salavatipour, M. R. 2003. Packing Steiner trees. In Proceeding of the 14th ACMSIAM Symposium on Discrete Algorithms. Google Scholar
Digital Library
- Jain, M. and Dovrolis, C. 2002. End-to-end available bandwidth: Measurement methodology, dynamics, and relation with TCP Throughput. In Proceedings of ACM SIGCOMM. Google Scholar
Digital Library
- Jannotti, J., Gifford, D. K., Johnson, K. L., Kaashoek, M. F., and James W. O'Toole, J. 2000. Overcast: Reliable multicasting with an overlay network. In Proceedings of the Conference on Operating Systems Design and Implementation (OSDI). Google Scholar
Digital Library
- Katabi, D., Handley, M., and Rohrs, C. 2002. Internet congestion control for high bandwidth-delay product networks. In Proceedings of ACM SIGCOMM. Google Scholar
Digital Library
- Kim, M. S., Lam, S. S., and Lee, D.-Y. 2002. Optimal distribution tree for Internet streaming media. Tech. rep. TR-02-48. Department of Computer Sciences, University of Texas at Austin, Austin, TX.Google Scholar
- Kostić, D., Braud, R., Killian, C., Vandekieft, E., Anderson, J. W., Snoeren, A. C., and Vahdat, A. 2005. Maintaining high bandwidth under dynamic network conditions. In Proceedings of the USENIX 2005 Annual Technical Conference. Google Scholar
Digital Library
- Kostić, D., Rodriguez, A., Albrecht, J., Bhirud, A., and Vahdat, A. 2003a. Using random subsets to build scalable network services. In Proceedings of the USENIX Symposium on Internet Technologies and Systems. Google Scholar
Digital Library
- Kostić, D., Rodriguez, A., Albrecht, J., and Vahdat, A. 2003b. Bullet: High bandwidth data dissemination using an overlay mesh. In Proceedings of the 19th ACM Symposium on Operating System Principles. Google Scholar
Digital Library
- Krohn, M. N., Freedman, M. J., and Mazieres, D. 2004. On-the-fly verification of rateless erasure codes for efficient content distribution. In Proceedings of the IEEE Symposium on Security and Privacy (Oakland, CA).Google Scholar
- Legout, A., Urvoy-Keller, G., and Michiardi, P. 2006. Rarest first and choke algorithms are enough. In IMC'06: Proceedings of the 6th ACM SIGCOMM Conference on Internet Measurement. 203--216. Google Scholar
Digital Library
- Luby, M. 2002. LT codes. In Proceedings of 43rd Annual IEEE Symposium on Foundations of Computer Science. Google Scholar
Digital Library
- Luby, M. G., Mitzenmacher, M., Shokrollahi, M. A., Spielman, D. A., and Stemann, V. 1997. Practical loss-resilient codes. In Proceedings of the 29th Annual ACM Symposium on the Theory of Computing (STOC '97). Ass. Comp. Mach. Press, New York, 150-- 159. Google Scholar
Digital Library
- Massoulie, L. and Vojnovic, M. 2005. Coupon replication systems. In Proceedings of ACM SIGMETRICS. Google Scholar
Digital Library
- Maymounkov, P. and Mazieres, D. 2003. Rateless codes and big downloads. In Proceedings of the Second International Peer to Peer Symposium (IPTPS).Google Scholar
- Padhye, J., Firoiu, V., Towsley, D., and Krusoe, J. 1998. Modeling TCP throughput: A simple model and its empirical validation. In Proceedings of ACM SIGCOMM (Vancouver, Canada). 303--314. Google Scholar
Digital Library
- Padmanabhan, V. N., Qiu, L., and Wang, H. J. 2003. Server-based inference of Internet link lossiness. In Proceedings of the IEEE INFOCOM. (San Francisco, CA).Google Scholar
- Padmanabhan, V. N., Wang, H. J., and Chou, P. A. 2003b. Resilient peer-to-peer streaming. In Proceedings of the 11th ICNP (Atlanta, GA). Google Scholar
Digital Library
- Padmanabhan, V. N., Wang, H. J., Chou, P. A., and Sripanidkulchai, K. 2002. Distributing streaming media content using cooperative networking. In Proceedings of ACM/IEEE NOSSDAV. Google Scholar
Digital Library
- Pai, V., Kumar, K., Tamilmani, K., Sambamurthy, V., and Mohr, A. E. 2005. Chainsaw: Eliminating trees from overlay multicast. In Proceedings of I PT PS. Google Scholar
Digital Library
- Park, K. and Pai, V. S. 2006. Scale and performance in the CoBlitz large-file distribution service. In Proceedings of the 3rd USENIX/ACM Symposium on Networked Systems Design and Implementation (NSDI). Google Scholar
Digital Library
- Peterson, L., Anderson, T., Culler, D., and Roscoe, T. 2002. A blueprint for introducing disruptive technology into the Internet. In Proceedings of ACM HotNets-I.Google Scholar
- Piatek, M., Isdal, T., Anderson, T., Krishnamurthy, A., and Venkataramani, A. 2007. Do incentives build robustness in BitTorrent? In Proceedings of the 4th USENIX/ACM Symposium on Networked Systems Design and Implementation (NSDI). Google Scholar
Digital Library
- Prim, R. C. 1957. Shortest connection networks and some generalizations. In Bell Syst. Tech. J. 36, 6, 1389--1401.Google Scholar
Cross Ref
- Qiu, D. and Srikant, R. 2004. Modeling and performance analysis of Bittorrentlike peer-to-peer networks. In Proceedings of ACM SIGCOMM. Google Scholar
Digital Library
- Rodriguez, A., Killian, C., Bhat, S., Kostić, D., and Vahdat, A. 2004a. MACEDON: Methodology for automatically creating, evaluating, and designing overlay networks. In Proceedings of the USENIX/ACM Symposium on Networked Systems Design and Implementation (NSDI, San Francisco, CA). Google Scholar
Digital Library
- Rodriguez, A., Kostić, D., and Vahdat, A. 2004b. Scalability in adaptive multi-metric overlays. In Proceedings the 24th International Conference on Distributed Computing Systems (ICDCS). Google Scholar
Digital Library
- Rowstron, A., Kermarrec, A.-M., Castro, M., and Druschel, P. 2001. SCRIBE: The design of a large-scale event notification infrastructure. In Proceedings of the Third International Workshop on Networked Group Communication. Google Scholar
Digital Library
- Saroiu, S., Gummadi, K. P., Dunn, R. J., Gribble, S. D., and Levy, H. M. 2002. An analysis of Internet content delivery systems. In Proceedings of 5th Symposium on Operating Systems Design and Implementation (OSDI). Google Scholar
Digital Library
- Savage, S., Collins, A., Hoffman, E., Snell, J., and Anderson, T. 1999. The end-to-end effects of Internet path selection. In Proceedings of ACM SIGCOMM. Google Scholar
Digital Library
- Sherwood, R., Braud, R., and Bhattacharjee, B. 2004. Slurpie: A cooperative bulk data transfer protocol. In Proceedings of IEEE INFOCOM.Google Scholar
- Shokrollahi, A. 2003. Raptor codes. Tech. rep. DF2003-06-001. Digital Fountain, Inc.Google Scholar
- Snoeren, A. C., Conley, K., and Gifford, D. K. 2001. Mesh-based content routing using XML. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP). Google Scholar
Digital Library
- Sung, Y.-W., Bishop, M., and Rao, S. 2006. Enabling contribution awareness in an overlay broadcasting system. In Proceedings of ACM SIGCOMM. Google Scholar
Digital Library
- Vahdat, A., Yocum, K., Walsh, K., Mahadevan, P., Kostić, D., Chase, J., and Becker, D. 2002. Scalability and accuracy in a large-scale network emulator. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI). Google Scholar
Digital Library
- Wang, L., Park, K., Pang, R., Pai, V. S., and Peterson, L. 2004. Reliability and Security in the CoDeeN content distribution network. In Proceedings of the USENIX 2004 Annual Technical Conference. Google Scholar
Digital Library
- Young, A., Chen, J., Ma, Z., Krishnamurthy, A., Peterson, L., and Wang, R. Y. 2004. Overlay mesh construction using interleaved spanning trees. In Proceedings of IEEE INFOCOM.Google Scholar
- Zhang, X., Liuy, J., Liz, B., and Yum, T.-S. P. 2005. CoolStreaming/DONet: A data-driven overlay network for efficient live media streaming. In Proceedings of IEEE INFOCOM.Google Scholar
Index Terms
High-bandwidth data dissemination for large-scale distributed systems
Recommendations
Bullet: high bandwidth data dissemination using an overlay mesh
SOSP '03: Proceedings of the nineteenth ACM symposium on Operating systems principlesIn recent years, overlay networks have become an effective alternative to IP multicast for efficient point to multipoint communication across the Internet. Typically, nodes self-organize with the goal of forming an efficient overlay tree, one that meets ...
Bullet: high bandwidth data dissemination using an overlay mesh
SOSP '03In recent years, overlay networks have become an effective alternative to IP multicast for efficient point to multipoint communication across the Internet. Typically, nodes self-organize with the goal of forming an efficient overlay tree, one that meets ...






Comments