article

Wide-area cooperative storage with CFS

Authors:

M. Frans Kaashoek,

Ion StoicaAuthors Info & Claims

ACM SIGOPS Operating Systems Review, Volume 35, Issue 5

Pages 202 - 215

https://doi.org/10.1145/502059.502054

Published: 21 October 2001 Publication History

Abstract

The Cooperative File System (CFS) is a new peer-to-peer read-only storage system that provides provable guarantees for the efficiency, robustness, and load-balance of file storage and retrieval. CFS does this with a completely decentralized architecture that can scale to large systems. CFS servers provide a distributed hash table (DHash) for block storage. CFS clients interpret DHash blocks as a file system. DHash distributes and caches blocks at a fine granularity to achieve load balance, uses replication for robustness, and decreases latency with server selection. DHash finds blocks using the Chord location protocol, which operates in time logarithmic in the number of servers.CFS is implemented using the SFS file system toolkit and runs on Linux, OpenBSD, and FreeBSD. Experience on a globally deployed prototype shows that CFS delivers data to clients as fast as FTP. Controlled tests show that CFS is scalable: with 4,096 servers, looking up a block of data involves contacting only seven servers. The tests also demonstrate nearly perfect robustness and unimpaired performance even when as many as half the servers fail.

References

[1]

Akamai Technologies, Inc. http://www.akamai.com/, 2001. Cambridge, MA.]]

[2]

ANDERSEN, n ., BALAKRISHNAN, H., KAASHOEK, M. F., AND MORRIS, R. Resilient overlay networks. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (Oct. 2001).]]

Digital Library

[3]

CHANKHUNTHOD, A., DANZIG, P., NEERDAELS, C., SCHWARTZ, M., AND WORRELL, K. k hierarchical lntemet object cache. In Proc. Usenix Technical Conference (Jan. 1996), pp. 153-163.]]

Digital Library

[4]

CHAUM, D. Untraceable electronic mail, return addresses, and digital pseudonyms. Communications of the ACM 24, 2 (Feb. 1981), 84-88.]]

Digital Library

[5]

CLARKE, I. A distributed decentralised information storage and retrieval system. Master's thesis, University of Edinburgh, 1999.]]

[6]

CLARKE, 1., SANDBERG, O., WILEY, B., AND HONG, T. Freenet: A distributed anonymous information storage and retrieval system. In Proceedings of the Workshop on Design Issues in Anonymity and Unobservability (July 2000), pp. 46-66.]]

Digital Library

[7]

DINGLEDINE, R., FREEDMAN, i ., AND MOLNAR, D. The Free Haven project: Distributed anonymous storage service. In Proceedings of the Workshop on Design Issues in Anonymity and Unobservability (July 2000), pp. 67-95.]]

Digital Library

[8]

FAN, L., CAO, P., ALMEIDA, J., AND BRODER, A. Summary cache: a scalable wide-area web-cache sharing protocol. Tech. Rep. 1361, Computer Science Department, University of Wisconsin, Madison, Feb. 1998.]]

Digital Library

[9]

FU, K., KAASHOEK, i . F., AND MAZIERES, D. Fast and secure distributed read-only file system. In Proceedings of the 4th USENIX Symposium on Operating Systems Design and Implementation (OSDI) (October 2000), pp. 181-196.]]

Digital Library

[10]

GADDE, S., CHASE, J., AND RABINOVICH, i . A taste of crispy squid. In Workshop on lnternet Server Performance (June 1998), pp. 129-136.]]

[11]

Gnutella website, http://gnutella.wego.com.]]

[12]

KARGER, D., LEHMAN, E., LEIGHTON, T., LEVINE, i, LEWIN, n., AND PANIGRAHY, R. Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the world wide web. In Proceedings of the 29th Annual ACM Symposium on Theory of Computing (May 1997), pp. 654-663.]]

Digital Library

[13]

KUBIATOWICZ, J., BINDEL, D., CHEN, Y., CZERWINSKI, S., EATON, P., GEELS, D, GUMMADI, R., RHEA, S., WEATHERSPOON, I-l., WEIMER, W., WELLS, C. AND ZHAO, B. Oceanstnre: An architecture for global-scale persistent storage, in Praceeedings of the Ninth international Conference on Architectural Support for Frograraming Languages and Operating Systems (ASPLOS) (November 2000), pp. 190-201.]]

Digital Library

[14]

LEWlN, D. Consistent hashing and random aces: Algorithms for caching in distributed networks. Master's thesis, MIT, 1998.]]

[15]

MALPANI, R., LURCH, J., AND BERGER, D. Making world wide web caching servers cooperate. In Fourth International World Wide Web Conference (1995), pp. 107-110.]]

[16]

MAZIERES, D. A toolkit for user-level file systems. In Proc. Usenix Technical Conference (June 2001), pp. 261-274.]]

Digital Library

[17]

MAZlERES, D., KAMINSKY, M., KAASHOEK, M. F., AND WITCHEL, E. Separating key management from file system security. In Proceedings of the 17th ACM Symposium on Operating Systems Principles (SOSP) (Dee. 1999), pp. 124-139.]]

Digital Library

[18]

MERKLE, R. C. A digital signature based on a conventional encryptioa function. In Advances in Cryptolagy--CRYPTO '87 (Berlin, 1987), C. Pomerance, Ed, vol. 293 of Lecture Notes in Computer Science, Springer-Vedag, pp. 369-378.]]

Digital Library

[19]

Mojo nation documentation, http:l/www.mojonafion.neVducsL]]

[20]

Napster. http:/lwww.napster com.]]

[21]

NO, T. S. E., STOICA, I., AND ZHAN6, H. A waypoint service approach to connect heterogeneous internet address spaces, tn Prec. Usenix Technical Conference (June 2001), pp. 319-332.]]

Digital Library

[22]

Ohaha. http://www., ohaha, co:a/a,s igu. him1, as of June 17, 2001, the Ohaha application is no longer available.]]

[23]

ORAM, A., Ed. Peer-to-Peer: Harnessing the Power of Disruptive Computation. O'Reilly & Associates, 2001.]]

Digital Library

[24]

PLAXTON, C., RAJARAMAN, R., AND RICHA. A. Accessing nearby copies of replicated objects in a dislxibuted environment, in Proceedings of the ACM SPAA (June 1997), pp. 311-320.]]

Digital Library

[25]

RABId, M. Efficient dispersal of information for security, load balancing, and fault tolerance. Journal of the ACM 36, 2 (1989), 335-348.]]

Digital Library

[26]

RATNASAMY, S., FRANOS, P., HANDLE'C, M., KARP, R., AND SIENKER, S. A sealable content-addressable network. In Proc_ ACM SUIGCOMM (San Diego, 2001).]]

Digital Library

[27]

REITER, M., AND RUBIN, A. Crowds: Anonymity for web transactions. ACM Transactions on Information and System Security 1, 1 (Nov. 1998), 66--92,]]

Digital Library

[28]

ROWSTRON, A., AND DRUSCrtEL, P. Pastry: Scalable, distributed object location and muting for large-scale peer-to-peer systems In Proceedings of the 18th IFIP/ACM International Conference on Distributed Systems Platforms (MMdleware 2001) (Nov. 2001).]]

Digital Library

[29]

ROWSTRON, A., AND DRUSCHEL, P. Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility. In Proceedings of the lgth ACM Symposium on Operating Systems Principles (Oct. 2001).]]

Digital Library

[30]

SHERMAN, A., KARGER, D ., BERKHEIMIER, A., BOGSTAD, B., DHANIDINA, R., IWAMOTO, K., KIM, B., MATKINS, L., AND YERUSHALMI, Y. Web caeMng with consistent hashing. Computer Neavorks 31, 11-16 (May 1999), 1203-1213.]]

Digital Library

[31]

STOICA, I., MORRIS, R., KARGER, D., KAASHOEK. M, F., AND BALAKR1SHNAN, H. Chord: A scalable peer-to-peer lookup service for internet applications. In Proc. ACM S1GCOMM (San Diego, 2001).]]

Digital Library

[32]

STOICA, I., MORRIS, R., KARGER, D., KAASHOEK, M. F., AND BALAKRISHNAN, H. Chord: A scalable peer-to-peer lookup service for interact applications. Tech. Rep. TR-819, MIT, Cambridge, MA, March 2001.]]

[33]

TYAN, T. A case study of server selection. Master's thesis, MtT, Sept. 2001.]]

[34]

WALDMAN, M., ROBIN, A., AND CRANOR, L. F. Publius: A robust, tamper-evident, censorship-resistant, web publishing system. In Proc. 9th USENIX Security Symposium (August 2000), pp. 59-72.]]

Digital Library

[35]

ZHAO, B., KUBIATOW}CZ, J., AND JOSEPH, A. Tapestry; An inflastxucture for fatflt-tolerant wide-area location and routing. Teeh. Rep. UCB/CSD-01-114I, Computer Science Division, U, C. Berkeley, Apr. 2001.]]

Digital Library

Cited By

Zhang KSha EZhuge QXu R(2024)An efficient flattened index structure with lazy restructuring and hotness awarenessFuture Generation Computer Systems10.1016/j.future.2023.11.025153(139-153)Online publication date: Apr-2024
https://doi.org/10.1016/j.future.2023.11.025
Zhao SHua ZXia Y(2023)SecDFS: A Secure and Decentralized File System2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS60453.2023.00097(623-630)Online publication date: 17-Dec-2023
https://doi.org/10.1109/ICPADS60453.2023.00097
Kubo TKawakami T(2023)An Adaptive Virtual Node Management Method for Overlay Networks Based on Multiple Time IntervalsComplex, Intelligent and Software Intensive Systems10.1007/978-3-031-35734-3_22(222-232)Online publication date: 19-Jun-2023
https://doi.org/10.1007/978-3-031-35734-3_22
Show More Cited By

Index Terms

Wide-area cooperative storage with CFS
1. Information systems
  1. Information storage systems
    1. Storage management
2. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        File systems management
        Memory management

Recommendations

Wide-area cooperative storage with CFS
SOSP '01: Proceedings of the eighteenth ACM symposium on Operating systems principles

The Cooperative File System (CFS) is a new peer-to-peer read-only storage system that provides provable guarantees for the efficiency, robustness, and load-balance of file storage and retrieval. CFS does this with a completely decentralized architecture ...
xFS: A Wide Area Mass Storage File System
Storage reduction for distributed disk-based backup in storage area network

Comments

Information & Contributors

Information

Published In

cover image ACM SIGOPS Operating Systems Review

ACM SIGOPS Operating Systems Review Volume 35, Issue 5

Dec. 2001

243 pages

ISSN:0163-5980

DOI:10.1145/502059

Issue’s Table of Contents

SOSP '01: Proceedings of the eighteenth ACM symposium on Operating systems principles
October 2001
254 pages
ISBN:1581133898
DOI:10.1145/502034
General Chair:
Keith Marzullo
University of California, San Diego
,
Program Chair:
M. Satyanarayanan
Carnegie Mellon University and Intel Research, Pittsburgh

Copyright © 2001 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2001

Published in SIGOPS Volume 35, Issue 5

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

711
Total Citations
View Citations
3,034
Total Downloads

Downloads (Last 12 months)117
Downloads (Last 6 weeks)10

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang KSha EZhuge QXu R(2024)An efficient flattened index structure with lazy restructuring and hotness awarenessFuture Generation Computer Systems10.1016/j.future.2023.11.025153(139-153)Online publication date: Apr-2024
https://doi.org/10.1016/j.future.2023.11.025
Zhao SHua ZXia Y(2023)SecDFS: A Secure and Decentralized File System2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS60453.2023.00097(623-630)Online publication date: 17-Dec-2023
https://doi.org/10.1109/ICPADS60453.2023.00097
Kubo TKawakami T(2023)An Adaptive Virtual Node Management Method for Overlay Networks Based on Multiple Time IntervalsComplex, Intelligent and Software Intensive Systems10.1007/978-3-031-35734-3_22(222-232)Online publication date: 19-Jun-2023
https://doi.org/10.1007/978-3-031-35734-3_22
Sun HWang QYue YZhao YFu S(2022)A storage computing architecture with multiple NDP devices for accelerating compaction performance in LSM-tree based KV storesJournal of Systems Architecture10.1016/j.sysarc.2022.102681(102681)Online publication date: Jul-2022
https://doi.org/10.1016/j.sysarc.2022.102681
Zhou AYi BLuo L(2022)Tree-structured data placement scheme with cluster-aided top-down transmission in erasure-coded distributed storage systemsComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2021.108714204:COnline publication date: 26-Feb-2022
https://dl.acm.org/doi/10.1016/j.comnet.2021.108714
Yin STeng YHu NJia X(2020)Decentralization of DNSProceedings of the 2020 International Conference on Cyberspace Innovation of Advanced Technologies10.1145/3444370.3444594(335-341)Online publication date: 4-Dec-2020
https://dl.acm.org/doi/10.1145/3444370.3444594
Vitale FChen JOdom WMcGrenere JWakkary RAndersen KOdom WDesjardins APetersen M(2020)Data DashboardProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395457(311-326)Online publication date: 3-Jul-2020
https://dl.acm.org/doi/10.1145/3357236.3395457
Wang DHu XXue QLi ZXu LWang W(2020)Resource Scheduling Algorithm on Mobile P2P Distribution Networks2020 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)10.1109/ISPA-BDCloud-SocialCom-SustainCom51426.2020.00109(666-673)Online publication date: Dec-2020
https://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom51426.2020.00109
Singh MKumar CNath P(2020)Finger Forwarding Scheme to Reduce Lookup Cost in Structured P2P NetworksWireless Personal Communications10.1007/s11277-020-07475-zOnline publication date: 17-May-2020
https://doi.org/10.1007/s11277-020-07475-z
Bozorgi AJadidi MAnderson J(2020)Challenges in Designing a Distributed Cryptographic File SystemSecurity Protocols XXVII10.1007/978-3-030-57043-9_17(177-192)Online publication date: 21-Aug-2020
https://doi.org/10.1007/978-3-030-57043-9_17
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents