Abstract
Contributory applications allow users to donate unused resources on their personal computers to a shared pool. Applications such as [email protected], [email protected], and Freenet are now in wide use and provide a variety of services, including data processing and content distribution. However, while several research projects have proposed contributory applications that support peer-to-peer storage systems, their adoption has been comparatively limited. We believe that a key barrier to the adoption of contributory storage systems is that contributing a large quantity of local storage interferes with the principal user of the machine.
To overcome this barrier, we introduce the Transparent File System (TFS). TFS provides background tasks with large amounts of unreliable storage—all of the currently available space—without impacting the performance of ordinary file access operations. We show that TFS allows a peer-to-peer contributory storage system to provide 40% more storage at twice the performance when compared to a user-space storage mechanism. We analyze the impact of TFS on replication in peer-to-peer storage systems and show that TFS does not appreciably increase the resources needed for file replication.
- Adya, A., Bolosky, W. J., Castro, M., Cermak, G., Chaiken, R., Douceur, J. R., Howell, J., Lorch, J. R., Theimer, M., and Wattenhofer, R. 2002. FARSITE: Federated, available, and reliable storage for an incompletely trusted environment. In 5th USENIX, Symposium on Operating System Design and Implementation (OSDI), Boston, MA. Google Scholar
Digital Library
- Bhagwan, R., Tati, K., Cheng, Y.-C., Savage, S., and Voelker, G. 2004. Total recall: System support for automated availability management. In Proceedings of the 1st ACM/USENIX Symposium on Networked Systems Design and Implementation (NSDI), San Jose, CA, 73--86. Google Scholar
Digital Library
- Blake, C. and Rodrigues, R. 2003. High availability, scalable storage, dynamic peer networks: Pick two. In 9th Workshop on Hot Topics in Operating Systems (HotOS), Lihue, HI, 1--6. Google Scholar
Digital Library
- Bolosky, W. J., Douceur, J. R., Ely, D., and Theimer, M. 2000. Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), ACM Press, New York, 34--43. Google Scholar
Digital Library
- Card, R., Ts'o, T., and Tweedie, S. 1994. Design and implementation of the second extended filesystem. In Proceedings of the 1st Dutch International Symposium on Linux, Laboratoire MASI, Institut Blaise Pascal and Massachussets Institute of Technology and University of Edinburgh.Google Scholar
- Chu, J., Labonte, K., and Levine, B. N. 2002. Availability and locality measurements of peer-to-peer file systems. In Proceedings of the 2nd Conference on Scalability and Traffic Control in IP Networks (ITCom).Google Scholar
- Cipar, J., Corner, M. D., and Berger, E. D. 2006. Transparent contribution of memory. In USENIX Annual Technical Conference (USENIX), 109--114. Google Scholar
Digital Library
- Clarke, I., Sandberg, O., Wiley, B., and Hong, T. 2001. Freenet: A distributed anonymous information storage and retrieval system. In Proceedings of the International Workshop on Design Issues in Anonymity and Unobservability, H. Federrath, ed. Lecture Notes in Computer Science, Vol. 2009, Springer. Google Scholar
Digital Library
- Dabek, F., Kaashoek, M. F., Karger, D., Morris, R., and Stoica, I. 2001. Wide-Area cooperative storage with CFS. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP), Chateau Lake Louise, Banff, Canada. Google Scholar
Digital Library
- Douceur, J. R. and Bolosky, W. J. 1999. A large-scale study of file-system contents. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), ACM Press, New York, 59--70. Google Scholar
Digital Library
- Freedman, M. J., Freudenthal, E., and Mazieres, D. 2004. Democratizing content publication with Coral. In Proceedings of the 1st USENIX Symposium on Networked Systems Design and Implementation (NSDI), San Francisco, CA. Google Scholar
Digital Library
- Freenet. 2007. Freenet FAQ. http://freenetproject.org/faq.html.Google Scholar
- Goedfrey, P. B., Shenker, S., and Stoica, I. 2006. Minimizing churn in distributed systems. In Proceedings of the ACM SIGCOMM, Data Communications Festival. Google Scholar
Digital Library
- Guha, S., Daswani, N., and Jain, R. 2006. An experimental study of the Skype peer-to-peer VoIP system. In Proceedings of the 5th International Workshop on Peer-to-Peer Systems (IPTPS), Santa Barbara, CA.Google Scholar
- Howard, J. H., Kazar, M. L., Menees, S. G., Nichols, D. A., Satyanarayanana, M., Sidebotham, R. N., and West, M. J. 1988. Scale and performance in a distributed file system. ACM Trans. Comput. Syst. 6, 1 (Feb.), 51--81. Google Scholar
Digital Library
- Huang, H., Hung, W., and Shin, K. G. 2005. FS2: Dynamic data replication in free disk space for improving disk performance and energy consumption. In Proceedings of the 20th ACM Symposium on Operating System Principles (SOSP). Google Scholar
Digital Library
- Larson, S. M., Snow, C. D., Shirts, M., and Pande, V. S. 2002. Computational Genomics. Horizon. Folding@Home and Genome@Home: Using distributed computing to tackle previously intractable problems in computational biology.Google Scholar
- Leonard, O. C., Neigh, J., Zadok, E., Osborn, J., Shater, A., and Wright, C. 2002. The design and implementation of elastic quotas. Tech. Rep. CUCS-014-02, Columbia University. June.Google Scholar
- Lumb, C. R., Schindler, J., and Ganger, G. R. 2002. Freeblock scheduling outside of disk firmware. In Proceedings of the Conference on File and Storage Technologies (FAST), 275--288. Google Scholar
Digital Library
- Mahajan, R., Castro, M., and Rowstron, A. 2003. Controlling the cost of reliability in peer-to-peer overlays. In Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS).Google Scholar
- McKusick, M. K., Joy, W. N., Leffler, S. J., and Fabry, R. S. 1984. A fast file system for UNIX. Comput. Syst. 2, 3, 181--197. Google Scholar
Digital Library
- Mickens, J. and Noble, B. D. 2006. Exploiting availability prediction in distributed systems. In Proceedings of the ACM/USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google Scholar
Digital Library
- Microsoft Corporation. 2007. http://www.microsoft.com/technet/prodtechnol/winxppro/reskit/c28621675.mspx.Google Scholar
- Morris, R., Karger, D., Kaashoek, F., and Balakrishnan, H. 2001. Chord: A scalable peer-to-peer lookup service for Internet applications. In the ACM SIGCOMM Data Communications Festival, San Diego, CA. Google Scholar
Digital Library
- Ousterhout, J. K. 1990. Why aren't operating systems getting faster as fast as hardware? In Proceedings of the USENIX Summer Conference, 247--256.Google Scholar
- Peterson, Z. and Burns, R. 2005. Ext3cow: A time-shifting file system for regulatory compliance. ACM Trans. Storage 1, 2, 190--212. Google Scholar
Digital Library
- Rhea, S., Geels, D., Roscoe, T., and Kubiatowicz, J. 2003. Handling churn in a DHT. Tech. Rep. UCB/CSD-03-1299, EECS Department, University of California, Berkeley.Google Scholar
- Rodrigues, R. and Liskov, B. 2005. High availability in DHTs: Erasure coding vs. replication. In Proceedings of the 4th International Workshop on Peer-to-Peer Systems (IPTPS). Google Scholar
Digital Library
- Rowstron, A. and Druschel, P. 2001a. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In Proceedings of the Middleware Conference.Google Scholar
- Rowstron, A. and Druschel, P. 2001b. Storage management and caching in past, a large-scale, persistent peer-to-peer storage utility. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP), Chateau Lake Louise, Banff, Canada. Google Scholar
Digital Library
- Seltzer, M., Bostic, K., McKusick, M. K., and Staelin, C. 1993. An implementation of a log-structured file system for UNIX. In the Winter USENIX Technical Conference. Google Scholar
Digital Library
- Smith, K. and Seltzer, M. 1997. File system aging. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, Seattle, WA. Google Scholar
Digital Library
- Stutzbach, D. and Rejaie, R. 2004. Towards a better understanding of churn in peer-to-peer networks. Tech. Rep. UO-CIS-TR-04-06, Department of Computer Science, University of Oregon. November.Google Scholar
- Venkataramani, A., Kokku, R., and Dahlin, M. 2002. TCP Nice: A mechanism for background transfers. SIGOPS Oper. Syst. Rev. 36, SI, 329--343. Google Scholar
Digital Library
Index Terms
Contributing storage using the transparent file system
Recommendations
Tuning file system block addressing for performance
ACM-SE 44: Proceedings of the 44th annual Southeast regional conferenceIn most general purpose file systems, data blocks are scattered throughout the disk so as not to require arbitrary chunks of contiguous disk space. To be able to find the nth data block in a file, both an index and indexing function must exist. The ...
Storage virtualization using a block-device file system
The design and organization of modern file systems has been traditionally driven by practical considerations related to the physical properties of computer disks Storage virtualization makes such considerations largely irrelevant, and file-system ...
Storage Devices, Local File System and Crossbar Network File System Characteristics, and 1 Terabyte File IO Benchmark on the "Numerical Simulator III"
MSS '03: Proceedings of the 20 th IEEE/11 th NASA Goddard Conference on Mass Storage Systems and Technologies (MSS'03)We benchmarked a mass storage system named "CeMSS" on the "Numerical Simulator III" System. It has eighty (80) RAID-5 disk arrays and forty (40) LTO tape drives, as a storage devices, and has an HSM based local file system and crossbar network file ...






Comments