skip to main content
research-article

CA-NFS: A congestion-aware network file system

Published:14 December 2009Publication History
Skip Abstract Section

Abstract

We develop a holistic framework for adaptively scheduling asynchronous requests in distributed file systems. The system is holistic in that it manages all resources, including network bandwidth, server I/O, server CPU, and client and server memory utilization. It accelerates, defers, or cancels asynchronous requests in order to improve application-perceived performance directly. We employ congestion pricing via online auctions to coordinate the use of system resources by the file system clients so that they can detect shortages and adapt their resource usage. We implement our modifications in the Congestion-Aware Network File System (CA-NFS), an extension to the ubiquitous network file system (NFS). Our experimental result shows that CA-NFS results in a 20% improvement in execution times when compared with NFS for a variety of workloads.

References

  1. Alonso, M. and Santonja, V. 1999. A new destage algorithm for disk cache: DOME. In Proceedings of the EUROMICRO Conference.Google ScholarGoogle Scholar
  2. Amir, Y., Awerbuch, B., Danilov, C., and Stanton, J. 2005. A cost-benefit flow control for reliable multicast and unicast in overlay networks. Trans. Netw. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Anderson, E., Spence, S., Swaminathan, R., Kallahalla, M., and Wang, Q. 2005. Quickly finding near-optimal storage designs. Trans. Comput. Syst. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Awerbuch, B., Azar, Y., Plotkin, S. A., and Waarts, O. 1994. Competitive routing of virtual circuits with unknown duration. In Proceedings of the Symposium on Discrete Algorithms. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Baker, M. G., Hartman, J. H., Kupfer, M. D., Shirriff, K. W., and Ousterhout, J. K. 1991. Measurements of a distributed file system. In Proceedings of the Symposium on Operating Systems Principles. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Batsakis, A., Burns, R., Kanevsky, A., Lentini, J., and Talpey, T. 2008. AWOL: An adaptive write optimizations in layer. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Capps, D. 2009. The IOzone Benchmark. http://www.iozone.com.Google ScholarGoogle Scholar
  8. Carson, S. D. and Setia, S. 1992. Analysis of the periodic update write policy for disk cache. Trans. Softw. Engin. 18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chambliss, D. D., Alvarez, G. A., Pandey, P., Jadav, D., Xu, J., Menon, R., and Lee, T. P. 2003. Performance virtualization for large-scale storage systems. In Proceedings of the Symposium on Reliable Distributed Systems. IEEE.Google ScholarGoogle Scholar
  10. Chun, B. N. and Culler, D. E. 2002. User-centric performance analysis of market-based cluster batch schedulers. In Proceedings of the IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Clearwater, S. 1996. Market-Based Control: A Paradigm for Distributed Resource Allocation World Scientific, Kirkkonummi, Finland. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Druschel, P. and Banga, G. 1996. Lazy receiver processing (LRP): A network subsystem architecture for server systems. In Proceedings of the Symposium on Operating Systems Design and Implementation. USENIX, Berkeley, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Fengguang, W. 2007. Adaptive read-ahead in the Linux kernel. http://lwn.net/Articles/155097/.Google ScholarGoogle Scholar
  14. Floyd, S. 1994. TCP and explicit congestion notification. Comput. Comm. Rev. 24, 5, 10--23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Floyd, S. and Jacobson, V. 1993. Random early detection gateways for congestion avoidance. Trans. Netw. 1, 4, 397--413. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Golding, R., Bosch, P., Staelin, C., Sullivan, T., and Wilkes, J. 1995. Idleness is not sloth. In Proceedings of the USENIX Annual Technical Conference. USENIX, Berkeley, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Goyal, P., Jadav, D., Modha, D. S., and Tewari, R. 2003. CacheCOW: QoS for storage system caches. In Proceedings of the International Workshop on Quality of Service (IWQoS'03). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Jacobson, V. 1988. Congestion avoidance and control. In Proceedings of the ACM SIGCOMM Communications Festival. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Jones, M. B., Rou, D., and Rou, M. 1997. Cpu reservations and time constraints: Efficient, predictable scheduling of independent activities. In Proceedings of the Symposium of Operating Systems and Principles. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Kaldewey, T., Wong, T., Golding, R., Povzner, A., Brandt, S. A., and Maltzahn, C. 2008. Virtualizing disk performance. In Proceedings of the Real-Time and Embedded Technology and Applications Symposium. IEEE, Los Alamitos, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Karlsson, M., Karamanolis, C., and Zhu, X. 2004. Triage: Performance isolation and differentiation for storage systems. In Proceedings of the International Workshop on Quality of Service.Google ScholarGoogle Scholar
  22. Kelly, F., Maulloo, A., and Tan, D. 1998. Rate control in communication networks: Shadow prices, proportional fairness and stability. J. Oper. Resear. Soc. 49.Google ScholarGoogle Scholar
  23. Ki, A. and Knowles, A. E. 1997. Adaptive data prefetching using cache information. In Proceedings of the International Conference on Supercomputing. IEEE, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Kustarz, E., Shepler, S., and A.Wilson. 2008. The new and improved filebench file system benchmarking framework. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA.Google ScholarGoogle Scholar
  25. Lai, K., Rasmusson, L., Adar, E., Zhang, L., and Huberman, B. A. 2005. Tycoon: An implementation of a distributed, market-based resource allocation system. Multiagent Grid Syst. 1, 3, 169--182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Levy, L., Blumrosen, L., and Nisan, N. 2001. On line markets for distributed object services: The majic system. In Proceedings of the USENIX Symposium on Internet Technologies and Systems (USITS'01). USENIX, Berkeley, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Li, X., Aboulnaga, A., Salem, K., Sachendina, A., and Gao, S. 2005. Second-tier cache management using write hints. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Lu, Y., Abdelzaher, T., Lu, C., and Tao, G. 2002. An adaptive control framework for qos guarantees and its application to differentiated caching services. In Proceedings of the International Workshop on Quality of Service. IEEE, Los Alamitos, CA.Google ScholarGoogle Scholar
  29. Lumb, C., Merchant, A., and Alvarez, G. 2003. Facade: Virtual storage devices with performance guarantees. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Massoulie, L. and Roberts, J. 1999. Bandwidth sharing: Objectives and algorithms. In Proceedings of the Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM). IEEE, Los Alamitos, CA.Google ScholarGoogle Scholar
  31. Mogul, J. 1994. A better update policy. In USENIX Summer Technical Conference. USENIX, Berkeley, CA, 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Nam, Y. J. and Park, C. 2002. An adaptive high-low water mark destage algorithm for cached RAID5. In Proceedings of the Pacific Rim Internataional Symposium on Dependable Computing. IEEE, Los Alamitos, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Nelson, M. N., Welch, B. B., and Ousterhout, J. K. 1988. Caching in the sprite network file system. Trans. Comput. Syst. 6, 1, 1--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Patterson, R. H., Gibson, G. A., Ginting, E., Stodolsky, D., and Zelenka, J. 1995. Informed prefetching and caching. In Proceedings of the ACM Symposium on Operating Systems Principles. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Povzner, A., Kaldewey, T., Brandt, S., Golding, R., Wong, T. M., and Maltzahn, C. 2008. Efficient guaranteed disk request scheduling with fahrrad. SIGOPS Oper. Syst. Rev. 42, 4, 13--25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Revel, D., McNamee, D., Steere, D., and Walpole, J. 1997. Adaptive prefetching for device independent file I/O. Tech. rep. CSE-97-005, School of Science and Engineering, Oregon Graduate Institute. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Riska, A., Riedel, E., and Iren, S. 2004. Adaptive disk scheduling for overload management. In Proceedings of the International Conference on the Quantitative Evaluation of Systems. IEEE, Los Alamitos, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Stoica, I., Abdel-Wahab, H., and Pothen, A. 1995. A Microeconomic Scheduler for Parallel Computers. In Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Strunk, J. D., Thereseka, E., Faloutsos, C., and Ganger, G. R. 2008. Using utility to provision storage systems. In Proceedings of the Conference on File and Storage Technologies. USENIX, Berkeley, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Uttamchandani, S., Yin, L., Alvarez, G. A., Palmer, J., and Agha, G. 2005. Chameleon: A self-evolving, fully-adaptive resource arbitrator for storage systems. In Proceedings of the USENIX Annual Technical Conference. USENIX, Berkeley, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Varma, A. and Jacobsen, Q. 1998. Destage algorithms for disk arrays with nonvolatile caches. Trans. Comput. 47, 2, 228--235. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Welsh, M., Culler, D., and Brewer, E. 2001. Seda: An architecture for well-conditioned scalable internet services. In Proceedings of the Symposium of Operating Systems Principles. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Zimmermann, Z. and Killat, U. 2002. Resource marking and fair rate allocation. In Proceedings of the International Conference on Communications. IEEE, Los Alamitos, CA, 1310--1314.Google ScholarGoogle Scholar

Index Terms

  1. CA-NFS: A congestion-aware network file system

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Storage
      ACM Transactions on Storage  Volume 5, Issue 4
      December 2009
      155 pages
      ISSN:1553-3077
      EISSN:1553-3093
      DOI:10.1145/1629080
      Issue’s Table of Contents

      Copyright © 2009 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 14 December 2009
      • Received: 1 August 2009
      • Accepted: 1 August 2009
      Published in tos Volume 5, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!