Gossiping over storage systems is practical

Abstract

Gossip-based mechanisms are touted for their simplicity, limited resource usage, robustness to failures, and tunable system behavior. These qualities make gossiping an ideal mechanism for storage systems that are responsible for maintaining and updating data in a mist of failures and limited resources (e.g., intermittent network connectivity, limited bandwidth, constrained communication range, or limited battery power). We focus on persistent storage systems that, unlike mere caches, are responsible for both the durability and the consistency of data. Examples of such systems may be encountered in many different environments, in particular: wide-area networks (constrained by limited bandwidth), wireless sensor networks (characterized by limited resources), and mobile ad hoc networks (suffering from intermittent connectivity). In this paper, we demonstrate the qualities of gossiping in these three respective environments.

References

1. Agency, N. S. Global Information Grid (GIG). http://www.nsa.gov/ia/industry/gig.cfm. May 2007.
2. Akdere, M., Bilgin, C. Ç., Korpeoglu, I., Ulusoy, O., and Çetintemel, U. A comparison of epidemic algorithms in wireless sensor networks. Computer Communications (Elsevier) 29 (2006), 2450--2457.
3. Akyildiz, I. F., Su, W., Sankarasubramaniam, Y., and Cayirci, E. A survey on sensor networks. IEEE Communications Magazine 40, 8 (2002), 102--114.
4. Bailey, N. T. J. The Mathematical Theory of Infectious Diseases and its Applications, second ed. Hafner Press, 1975.
5. Bavier, A., Bowman, M., Chun, B., Culler, D., Karlin, S., Muir, S., Peterson, L., Roscoe, T., Spalink, T., and Wawrzoniak, M. Operating system support for planetary-scale network services. In Proceedings of the 1st USENIX Symp. on Network Systems Design and Implementation (NSDI 2004) (2004).
6. Bernstein, P. A., and Goodman, N. Multiversion concurrency control-theory and algorithms. ACM Trans. on Database Systems 8, 4 (1983), 465--483.
7. Blake, C., and Rodrigues, R. High availability, scalable storage, dynamic peer networks: Pick two. In Proceedings of the 9th USENIX Workshop on Hot Topics in Operating Systems (HotOS IX) (2003), pp. 1--6.
8. Brewer, E. A. Towards robust distributed systems (invited talk). In The 19th ACM Symp. on Principles of Distributed Computing (PODC 2000) (2000).
9. Busnel, Y., Bertier, M., Fleury, E., and Kermarrec, A.-M. GCP : Gossip-based code propagation for large-scaled mobile wireless sensor network. Tech. rep., INRIA, 2007.
10. Chun, B., Dabek, F., Haeberlen, A., Sit, E., Weatherspoon, H., Kaashoek, M. F., Kubiatowicz, J., and Morris, R. Efficient replica maintenance for distributed storage systems. In Proceedings of the 3rd USENIX Symp. on Network Systems Design and Implementation (NSDI 2006) (2006).
11. Coan, B. A., Oki, B. M., and Kolodner, E. K. Limitations on database availability when networks partition. In Proceedings of the 5th ACM Symp. on Principles of Distributed Computing (PODC 1986) (1986), pp. 187--194.
12. Dabek, F. A Distributed Hash Table. PhD thesis, Massachusetts Institute of Technology, 2005.
13. Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D., and Terry, D. Epidemic algorithms for replicated database maintenance. In Proceedings of the 6th ACM Symp. on Principles of Distributed Computing (PODC 1987) (1987), pp. 1--12.
14. Drabkin, V., Friedman, R., Kliot, G., and Segal, M. RAPID: Reliable probabilistic dissemination in wireless ad-hoc networks. Tech. Rep. CS-2006-19, Computer Science Department, Technion -- Israel Institute of Technology, 2006.
15. Gavidia, D., Voulgaris, S., and van Steen, M. Epidemic-style monitoring in large-scale wireless sensor networks. Tech. Rep. IR-CS-012, Vrije Universiteit Amsterdam, 2005.
16. Ghose, A., Grossklags, J., and Chuang, J. Resilient data-centric storage in wireless sensor networks. IEEE Distributed Systems Online (2003).
17. Gilbert, S., and Lynch, N. Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33, 2 (2002), 51--59.
18. Haas, Z. J., Halpern, J. Y., and Li, L. Gossip-based ad hoc routing. In Proceedings of the 21st Annual Joint Conf. of the IEEE Computer and Communications Societies (INFOCOM 2002) (2002), pp. 1707--1716.
19. Hara, T. Effective replica allocation in ad hoc networks for improving data accessibility. In Proc. of the 20th Joint Conf. of the IEEE Computer and Communications Societies (INFOCOM 2001) (2001), pp. 1568--1576.
20. Hara, T. Replica allocation in ad hoc networks with periodic data update. In Proceedings of the 3rd IEEE Int'l Conf. on Mobile Data Management, (MDM 2002) (2002), pp. 79--86.
21. Herlihy, M., and Wing, J. M. Linearizability: A correctness condition for concurrent objects. ACM Trans. on Programming Language Systems (TOPLAS). 12, 3 (1990), 463--492.
22. Jelasity, M., and Babaoglu, O. T-Man: Fast gossip-based construction of large-scale overlay topologies. Tech. Rep. UBLCS-2004-7, University of Bologna, Department of Computer Science, 2004.
23. Johnson, D. B., Maltz, D. A., and Broch, J. Ad Hoc Networking. Addison-Wesley, 2001, ch. DSR: The Dynamic Source Routing Protocol for Multi-Hop Wireless Ad Hoc Networks, pp. 139--172.
24. Kermarrec, A.-M., Massouli, L., and Ganesh, A. J. Probabilistic reliable dissemination in large-scale systems. IEEE Trans. on Parallel and Distributed Systems 14, 3 (2003).
25. Luo, J., Eugster, P., and Hubaux, J.-P. Route driven gossip: probabilistic reliable multicast in ad hoc networks. In Proceedings of the 22nd Joint Conf. of the IEEE Computer and Communications Societies (INFOCOM 2003) (2003), pp. 2229--2239.
26. Luo, J., Eugster, P. T., and Hubaux, J.-P. Pilot: Probabilistic lightweight group communication system for ad hoc networks. IEEE Trans. on Mobile Computing 3, 2 (2004), 164--179.
27. Mainwaring, A., Polastre, J., Szewczyk, R., Culler, D., and Anderson, J. Wireless sensor networks for habitat monitoring. In Proceedings of the 1st ACM Int'l Workshop on Wireless Sensor Networks and Applications (WSNA 2002) (2002).
28. Mathur, G., Desnoyers, P., Ganesan, D., and Shenoy, P. Ultra-low power data storage for sensor networks. In Proceedings of the 2007 Conf. on Information Processing in Sensor Networks (IPSN) -- SPOTS Track (2006).
29. Miranda, H., Leggio, S., Rodrigues, L., and Raatikainen, K. A stateless neighbour-aware cooperative caching protocol for ad-hoc networks. DI/FCUL TR 05-23, Department of Informatics, University of Lisbon, 2005.
30. Morales, R., and Gupta, I. Providing both scale and security through a single core probabilstic protocol. In Proceedings of the Workshop on Stochasticity in Distributed Systems (StoDiS 2005) (2005).
31. Papadimitriou, C. H. Serializability of concurrent database updates. Journal of the ACM 24, 4 (1979), 631--653.
32. Peterson, L., Fiuczynski, A. B. E., and Muir, S. Experiences building planetlab. In Proceedings of the 7th USENIX Symp. on Operating Systems Design and Implementation (OSDI 2006) (2006).
33. Pittel, B. On spreading of a rumor. SIAM Journal of Applied Mathematics 47 (1987), 213--223.
34. Pleisch, S., Balakrishnan, M., Birman, K., and van Renesse, R. MISTRAL: Efficient flooding in mobile ad-hoc networks. In Proceedings of the 7th ACM Int'l Symp. on Mobile Ad Hoc Networking and Computing (MobiHoc 2006) (2006), pp. 1--12.
35. Ramabhadran, S., and Pasquale, J. Analysis of long-running replicated systems. In Proceedings of the 25th Joint Conf. of the IEEE Computer and Communications Societies (INFOCOM 2006) (2006).
36. Ratnasamy, S., Karp, B., Yin, L., Yu, F., Estrin, D., Govindan, R., and Shenker, S. GHT: a geographic hash table for data-centric storage. In Proceedings of the 1st ACM Int'l Workshop on Wireless Sensor Networks and Applications (WSNA 2002) (2002), pp. 78--87.
37. Sit, E., Haeberlen, A., Dabek, F., Chun, B.-G., Weatherspoon, H., Morris, R., Kaashoek, M. F., and Kubiatowicz, J. Proactive replication for data durability. In Proceedings of the 5th Int'l Workshop on Peer-to-Peer Systems (IPTPS 2006) (2006).
38. Terry, D. B., Theimer, M. M., Petersen, K., Demers, A. J., Spreitzer, M. J., and Hauser, C. H. Managing update conflicts in bayou, a weakly connected replicated storage system. In Proceedings of the 15th ACM Symp. on Operating Systems Principles (SOSP 1995) (1995), pp. 172--183.
39. van Renesse, R. Power-aware epidemics. In Int'l Workshop on Reliable Peer-to-Peer Systems (2002).
40. van Renesse, R., Birman, K. P., and Vogels, W. Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining. ACM Trans. on Computer Systems (TOCS) 21, 2 (2003), 164--206.

Index Terms

• Computer systems organization

  • Architectures

    • Distributed architectures

• Software and its engineering

  • Software organization and properties

    • Software system structures

      • Distributed systems organizing principles