Abstract
Peer to peer backup systems store data on “unreliable” peers that can leave the system at any moment. In this case, the only way to assure durability of the data is to add redundancy using either replication or erasure codes. Erasure codes are able to provide the same reliability as replication requiring much less storage space. Erasure coding breaks the data into blocks that are encoded and then stored on different nodes. However, when storage nodes permanently abandon the system, new redundant blocks must be created, which is referred to as repair. For “classical” erasure codes, generating a new block requires the transmission of k blocks over the network, resulting in a high repair traffic. Recently, two new classes of erasure codes, Regenerating Codes and Hierarchical Codes, have been proposed that significantly reduce the repair traffic. Regenerating Codes reduce the amount of data uploaded by each peer involved in the repair, while Hierarchical Codes reduce the number of nodes participating in the repair. In this article we propose to combine these two codes to devise a new class of erasure codes called ER-Hierarchical Codes that combine the advantages of both.
- Adya, A., Bolosky, W., Castro, M., Cermak, G., Chaiken, R., Douceur, J., Howell, J., Lorch, J., Theimer, M., and Wattenhofer, R. 2002. Farsite: Federated, available and reliable storage for an incompletely trusted environment. In Proceedings of the 5th USENIX Symposium on Operating Systems Design and Implementation. 1--14. Google Scholar
Digital Library
- Blake, C. and Rodrigues, R. 2003. High availability, scalable storage, dynamic peer networks: Pick two. In Proceedings of 9th the Usenix Workshop on Hot Topics in Operating Systems. Lihue, Hawaii, 1--6. Google Scholar
Digital Library
- Chun, B.-G., Dabek, F., Haeberlen, A., Sit, E., Weatherspoon, H., Kaashoek, M., and Kubiatowicz, J. 2006a. Efficient replica maintenance for distributed storage systems. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation. Google Scholar
Digital Library
- Chun, B.-G., Dabek, F., Haeberlen, A., Sit, E., Weatherspoon, H., M. Frans Kaashoek, J. K., and Morris, R. 2006b. Proactive replication for data durability. In Proceedings of the USENIX International Workshop on Peer-to-Peer Systems.Google Scholar
- Chun, B.-G., Ratnasamy, S., and Kohler, E. 2008. Netcomplex: A complexity metric for networked system designs. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation. 393--406. Google Scholar
Digital Library
- Dabek, F., Kaashoek, M. F., Karger, D. R., Morris, R., and Stoica, I. 2001. Wide-area cooperative storage with CFS. In Proceedings of the ACM Symposium on Operating Systems Principles. 202--215. Google Scholar
Digital Library
- Dimakis, A. G., Godfrey, P. B., Wu, Y., Wainwright, M. J., and Ramchandran, K. 2010a. Network coding for distributed storage systems. IEEE Trans. Inform. Theory 56, 9, 4539--4551. Google Scholar
Digital Library
- Dimakis, A. G., Ramchandran, K., Wu, Y., and Suh, C. 2010b. A survey on network codes for distributed storage. CoRR. http://arxiv.org/abs/1004.4438.Google Scholar
- Dischinger, M., Gummadi, K. P., Haeberlen, A., and Saroiu, S. 2007. Characterizing residential broadband networks. In Proceedings of the ACM Internet Measurement Conference (IMC). Google Scholar
Digital Library
- Druschel, P. and Rowstron, A. 2001. PAST: A large-scale, persistent peer-to-peer storage utility. In Proceedings of the USENIX Workshop on Hot Topics in Operating Systems. 75. Google Scholar
Digital Library
- Duminuco, A. 2009. Data redundancy and maintenance for peer-to-peer file backup systems. Ph.D. thesis, Telecom ParisTech.Google Scholar
- Duminuco, A. and Biersack, E. 2009a. A practical study of regenerating codes for peer-to-peer backup systems. In Proceedings of the IEEE International Conference on Distributed Computing Systems. 376--384. Google Scholar
Digital Library
- Duminuco, A. and Biersack, E. W. 2009b. Hierarchical codes: A flexible trade-off for erasure codes. J. Peer-to-Peer Netw. Appl. 2, 52--66.Google Scholar
- Duminuco, A., Biersack, E., and En-Najjary, T. 2007. Proactive replication in distributed storage systems using machine availability estimation. In Proceedings of the ACM Conference on emerging Networking Experiments and Technologies. Google Scholar
Digital Library
- Godfrey, B. 2006. Repository of Availability Traces. http://www.cs.berkeley.edu/ pbg/availability/.Google Scholar
- Haeberlen, A., Mislove, A., and Druschel, P. 2005. Glacier: Highly durable, decentralized storage despite massive correlated failures. In Proceedings of the 2nd USENIX Symposium on Networked Systems Design and Implementation. 143--158. Google Scholar
Digital Library
- HardDriver. 2010. Cost of hard drive storage space. http://ns1758.ca/winch/winchest.html.Google Scholar
- Kondo, D., Javadi, B., Iosup, A., and Epema, D. 2010. The failure trace archive: Enabling comparative analysis of failures in diverse distributed systems. Proceedings of the IEEE/ACM International Conference on Cluster, Cloud and Grid Computing. 398--407. Google Scholar
Digital Library
- Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., Wells, C., and Zhao, B. 2000. Oceanstore: An architecture for global-scale persistent storage. In Proceedings of the 9th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 190--201. Google Scholar
Digital Library
- Lin, W. K., Chiu, D. M., and Lee, Y. B. 2004. Erasure code replication revisited. In Proceedings of the IEEE International Conference on Peer-to-Peer Computing. 90--97. Google Scholar
Digital Library
- Mitzenmacher, M. 2004. Digital fountains: A survey and look forward. In Proceedings of the IEEE Information Theory Workshop. 271--276.Google Scholar
Cross Ref
- Pamies-Juarez, L. and Biersack, E. 2011. Cost analysis of redundancy schemes for distributed storage systems. CoRR. http://arxiv.org/abs/1103.2662.Google Scholar
- Pamies-Juarez, L., Garcia-Lopez, P., and Sanchez-Artigas, M. 2010. Availability and Redundancy in Harmony: Measuring Retrieval Times in P2P Storage Systems. In Proceedings of the IEEE International Conference on Peer-to-Peer Computing. 1--10.Google Scholar
- Plank, J. S. 2005. Erasure codes for storage applications. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’05).Google Scholar
- Plank, J. S. 2008. A new MDS erasure code for RAID-6. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’08).Google Scholar
- Rashmi, K. V., Shah, N. B., Kumar, P. V., and Ramchandran, K. 2009. Explicit construction of optimal exact regenerating codes for distributed storage. In Proceedings of the Annual Allerton Conference on Communication, Control, and Computing. 1243--1249. Google Scholar
Digital Library
- Rashmi, K. V., Shah, N. B., and Kumar, P. V. 2010. Optimal exact-regenerating codes for distributed storage at the msr and mbr points via a product-matrix construction. CoRR abs/1005.4178. http://arxiv.org/abs/1005.4178. Google Scholar
Digital Library
- Richardson, T. and Urbanke, R. 2008. Modern Coding Theory. Cambridge University Press, Cambridge, UK. Google Scholar
Digital Library
- Rodrigues, R. and Liskov, B. 2005. High availability in DHTs: Erasure coding vs. replication. In Proceedings of the USENIX International Workshop on Peer-To-Peer Systems. New York. Google Scholar
Digital Library
- Shah, N. B., V., R. K., Kumar, P. V., and Ramachandran, K. 2010. Interference alignment in regenerating codes for distributed storage: Necessity and code constructions. CoRR (submitted to the IEEE Trans. Inform. Theory). http://arxiv.org/abs/1005.1634.Google Scholar
- Shojania, H. and Li, B. 2009. Pushing the envelope: Extreme network coding on the GPU. In Proceedings of the 29th International Conference on Distributed Computing Systems (ICDCS). Google Scholar
Digital Library
- Suh, C. and Ramchandran, K. 2010. Exact regeneration codes for distributed storage repair using interference alignment. http://arxiv.org/abs/1001.0107.Google Scholar
- Weatherspoon, H. and Kubiatowicz, J. D. 2002. Erasure coding vs. replication: A quantitative comparison. In Proceedings of the USENIX International Workshop on Peer-To-Peer Systems. Google Scholar
Digital Library
- Wu, Y. and Dimakis, A. G. 2009. Reducing repair traffic for erasure coding-based storage via interference alignment. In Proceedings of the IEEE International Symposium on Information Theory. 2276--2280. Google Scholar
Digital Library
- Wuala. 2010. Official website. http://www.wuala.com.Google Scholar
- Zhang, Z., Deshpande, A., Ma, X., Thereska, E., and Narayanan, D. 2010. Does erasure coding have a role to play in my data center? Microsoft Research Tech. rep. MSR-TR-2010-52.Google Scholar
Index Terms
Reducing Repair Traffic in P2P Backup Systems: Exact Regenerating Codes on Hierarchical Codes
Recommendations
A Practical Study of Regenerating Codes for Peer-to-Peer Backup Systems
ICDCS '09: Proceedings of the 2009 29th IEEE International Conference on Distributed Computing SystemsIn distributed storage systems, erasure codes represent an attractive solution to add redundancy to stored data while limiting the storage overhead. They are able to provide the same reliability as replication requiring much less storage space. Erasure ...
Robust Redundancy Scheme for the Repair Process: Hierarchical Codes in the Bandwidth-Limited Systems
High performance computing can be well supported by the Grid or cloud computing systems. However, these systems have to overcome the failure risks, where data is stored in the "unreliable" storage nodes that can leave the system at any moment and the ...
An overview of codes tailor-made for better repairability in networked distributed storage systems
The increasing amount of digital data generated by today's society asks for better storage solutions. This survey looks at a new generation of coding techniques designed specifically for the maintenance needs of networked distributed storage systems (...






Comments