skip to main content
article

Contributing storage using the transparent file system

Published:01 October 2007Publication History
Skip Abstract Section

Abstract

Contributory applications allow users to donate unused resources on their personal computers to a shared pool. Applications such as [email protected], [email protected], and Freenet are now in wide use and provide a variety of services, including data processing and content distribution. However, while several research projects have proposed contributory applications that support peer-to-peer storage systems, their adoption has been comparatively limited. We believe that a key barrier to the adoption of contributory storage systems is that contributing a large quantity of local storage interferes with the principal user of the machine.

To overcome this barrier, we introduce the Transparent File System (TFS). TFS provides background tasks with large amounts of unreliable storage—all of the currently available space—without impacting the performance of ordinary file access operations. We show that TFS allows a peer-to-peer contributory storage system to provide 40% more storage at twice the performance when compared to a user-space storage mechanism. We analyze the impact of TFS on replication in peer-to-peer storage systems and show that TFS does not appreciably increase the resources needed for file replication.

References

  1. Adya, A., Bolosky, W. J., Castro, M., Cermak, G., Chaiken, R., Douceur, J. R., Howell, J., Lorch, J. R., Theimer, M., and Wattenhofer, R. 2002. FARSITE: Federated, available, and reliable storage for an incompletely trusted environment. In 5th USENIX, Symposium on Operating System Design and Implementation (OSDI), Boston, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bhagwan, R., Tati, K., Cheng, Y.-C., Savage, S., and Voelker, G. 2004. Total recall: System support for automated availability management. In Proceedings of the 1st ACM/USENIX Symposium on Networked Systems Design and Implementation (NSDI), San Jose, CA, 73--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Blake, C. and Rodrigues, R. 2003. High availability, scalable storage, dynamic peer networks: Pick two. In 9th Workshop on Hot Topics in Operating Systems (HotOS), Lihue, HI, 1--6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bolosky, W. J., Douceur, J. R., Ely, D., and Theimer, M. 2000. Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), ACM Press, New York, 34--43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Card, R., Ts'o, T., and Tweedie, S. 1994. Design and implementation of the second extended filesystem. In Proceedings of the 1st Dutch International Symposium on Linux, Laboratoire MASI, Institut Blaise Pascal and Massachussets Institute of Technology and University of Edinburgh.Google ScholarGoogle Scholar
  6. Chu, J., Labonte, K., and Levine, B. N. 2002. Availability and locality measurements of peer-to-peer file systems. In Proceedings of the 2nd Conference on Scalability and Traffic Control in IP Networks (ITCom).Google ScholarGoogle Scholar
  7. Cipar, J., Corner, M. D., and Berger, E. D. 2006. Transparent contribution of memory. In USENIX Annual Technical Conference (USENIX), 109--114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Clarke, I., Sandberg, O., Wiley, B., and Hong, T. 2001. Freenet: A distributed anonymous information storage and retrieval system. In Proceedings of the International Workshop on Design Issues in Anonymity and Unobservability, H. Federrath, ed. Lecture Notes in Computer Science, Vol. 2009, Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Dabek, F., Kaashoek, M. F., Karger, D., Morris, R., and Stoica, I. 2001. Wide-Area cooperative storage with CFS. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP), Chateau Lake Louise, Banff, Canada. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Douceur, J. R. and Bolosky, W. J. 1999. A large-scale study of file-system contents. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), ACM Press, New York, 59--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Freedman, M. J., Freudenthal, E., and Mazieres, D. 2004. Democratizing content publication with Coral. In Proceedings of the 1st USENIX Symposium on Networked Systems Design and Implementation (NSDI), San Francisco, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Freenet. 2007. Freenet FAQ. http://freenetproject.org/faq.html.Google ScholarGoogle Scholar
  13. Goedfrey, P. B., Shenker, S., and Stoica, I. 2006. Minimizing churn in distributed systems. In Proceedings of the ACM SIGCOMM, Data Communications Festival. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Guha, S., Daswani, N., and Jain, R. 2006. An experimental study of the Skype peer-to-peer VoIP system. In Proceedings of the 5th International Workshop on Peer-to-Peer Systems (IPTPS), Santa Barbara, CA.Google ScholarGoogle Scholar
  15. Howard, J. H., Kazar, M. L., Menees, S. G., Nichols, D. A., Satyanarayanana, M., Sidebotham, R. N., and West, M. J. 1988. Scale and performance in a distributed file system. ACM Trans. Comput. Syst. 6, 1 (Feb.), 51--81. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Huang, H., Hung, W., and Shin, K. G. 2005. FS2: Dynamic data replication in free disk space for improving disk performance and energy consumption. In Proceedings of the 20th ACM Symposium on Operating System Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Larson, S. M., Snow, C. D., Shirts, M., and Pande, V. S. 2002. Computational Genomics. Horizon. Folding@Home and Genome@Home: Using distributed computing to tackle previously intractable problems in computational biology.Google ScholarGoogle Scholar
  18. Leonard, O. C., Neigh, J., Zadok, E., Osborn, J., Shater, A., and Wright, C. 2002. The design and implementation of elastic quotas. Tech. Rep. CUCS-014-02, Columbia University. June.Google ScholarGoogle Scholar
  19. Lumb, C. R., Schindler, J., and Ganger, G. R. 2002. Freeblock scheduling outside of disk firmware. In Proceedings of the Conference on File and Storage Technologies (FAST), 275--288. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Mahajan, R., Castro, M., and Rowstron, A. 2003. Controlling the cost of reliability in peer-to-peer overlays. In Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS).Google ScholarGoogle Scholar
  21. McKusick, M. K., Joy, W. N., Leffler, S. J., and Fabry, R. S. 1984. A fast file system for UNIX. Comput. Syst. 2, 3, 181--197. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Mickens, J. and Noble, B. D. 2006. Exploiting availability prediction in distributed systems. In Proceedings of the ACM/USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Microsoft Corporation. 2007. http://www.microsoft.com/technet/prodtechnol/winxppro/reskit/c28621675.mspx.Google ScholarGoogle Scholar
  24. Morris, R., Karger, D., Kaashoek, F., and Balakrishnan, H. 2001. Chord: A scalable peer-to-peer lookup service for Internet applications. In the ACM SIGCOMM Data Communications Festival, San Diego, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Ousterhout, J. K. 1990. Why aren't operating systems getting faster as fast as hardware? In Proceedings of the USENIX Summer Conference, 247--256.Google ScholarGoogle Scholar
  26. Peterson, Z. and Burns, R. 2005. Ext3cow: A time-shifting file system for regulatory compliance. ACM Trans. Storage 1, 2, 190--212. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Rhea, S., Geels, D., Roscoe, T., and Kubiatowicz, J. 2003. Handling churn in a DHT. Tech. Rep. UCB/CSD-03-1299, EECS Department, University of California, Berkeley.Google ScholarGoogle Scholar
  28. Rodrigues, R. and Liskov, B. 2005. High availability in DHTs: Erasure coding vs. replication. In Proceedings of the 4th International Workshop on Peer-to-Peer Systems (IPTPS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Rowstron, A. and Druschel, P. 2001a. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In Proceedings of the Middleware Conference.Google ScholarGoogle Scholar
  30. Rowstron, A. and Druschel, P. 2001b. Storage management and caching in past, a large-scale, persistent peer-to-peer storage utility. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP), Chateau Lake Louise, Banff, Canada. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Seltzer, M., Bostic, K., McKusick, M. K., and Staelin, C. 1993. An implementation of a log-structured file system for UNIX. In the Winter USENIX Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Smith, K. and Seltzer, M. 1997. File system aging. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, Seattle, WA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Stutzbach, D. and Rejaie, R. 2004. Towards a better understanding of churn in peer-to-peer networks. Tech. Rep. UO-CIS-TR-04-06, Department of Computer Science, University of Oregon. November.Google ScholarGoogle Scholar
  34. Venkataramani, A., Kokku, R., and Dahlin, M. 2002. TCP Nice: A mechanism for background transfers. SIGOPS Oper. Syst. Rev. 36, SI, 329--343. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Contributing storage using the transparent file system

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!