skip to main content
article

Generalized file system dependencies

Published:14 October 2007Publication History
Skip Abstract Section

Abstract

Reliable storage systems depend in part on "write-before" relationships where some changes to stable storage are delayed until other changes commit. A journaled file system, for example, must commit a journal transaction before applying that transaction's changes, and soft updates and other consistency enforcement mechanisms have similar constraints, implemented in each case in system-dependent ways. We present a general abstraction, the patch, that makes write-before relationships explicit and file system agnostic. A patch-based file system implementation expresses dependencies among writes, leaving lower system layers to determine write orders that satisfy those dependencies. Storage system modules can examine and modify the dependency structure, and generalized file system dependencies are naturally exportable to user level. Our patch-based storage system, Feather stitch, includes several important optimizations that reduce patch overheads by orders of magnitude. Our ext2 prototype runs in the Linux kernel and supports a synchronous writes, soft updates-like dependencies, and journaling. It outperforms similarly reliable ext2 and ext3 configurations on some, but not all, benchmarks. It also supports unusual configurations, such as correct dependency enforcement within a loopback file system, and lets applications define consistency requirements without micromanaging how those requirements are satisfied.

Skip Supplemental Material Section

Supplemental Material

Video

References

  1. Dovecot. Version 1.0 beta7, http://www.dovecot.org/.Google ScholarGoogle Scholar
  2. Subversion. http://subversion.tigris.org/.Google ScholarGoogle Scholar
  3. UW IMAP toolkit. http://www.washington.edu/imap/.Google ScholarGoogle Scholar
  4. Burnett, N. C. Information and Control in File System Buffer Management. PhD thesis, University of Wisconsin--Madison, July 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Cornell, B., P.A. Dinda, and F.E. Bustamante. Wayback: A user-level versioning file system for Linux. In Proc. 2004 USENIX Annual Technical Conference, FREENIX Track, pages 19--28, June 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Crispin, M. Internet Message Access Protocol-version 4rev1. RFC 3501, IETF, Mar. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Denehy, T.E., A.C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. Journal-guided resynchronization for software RAID. In Proc. 4th USENIX Conference on File and Storage Technologies (FAST '05), pages 87--100, Dec. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Gal, E. and S. Toledo. A transactional Flash file system for microcontrollers. In Proc. 2005 USENIX Annual Technical Conference, pages 89--104, Apr. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ganger, G. R., M. K. McKusick, C. A. N. Soules, and Y. N. Patt. Soft updates: A solution to the metadata update problem in file systems. ACM Transactions on Computer Systems, 18 (2): 127--153, May 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Heidemann, J.S. and G. J. Popek. File-system development with stackable layers. ACM Transactions on Computer Systems, 12 (1): 58--89, Feb. 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Hitz, D., J. Lau, and M. Malcolm. File system design for an NFS file server appliance. In Proc. USENIX Winter 1994 Technical Conference, pages 235--246, Jan. 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Huang, H., W. Hung, and K. G. Shin. FS2: Dynamic data replication in free disk space for improving disk performance and energy consumption. In Proc. 20th ACM Symposium on Operating Systems Principles, pages 263--276, Oct. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Kaashoek, M. F., D. R. Engler, G. R. Ganger, H. M. Briceño, R. Hunt, D. Mazières, T. Pinckney, R. Grimm, J. Jannotti, and K. Mackenzie. Application performance and flexibility on Exokernel systems. In Proc. 16th ACM Symposium on Operating Systems Principles, pages 52--65, Oct. 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Katcher, J. PostMark: A new file system benchmark. Technical Report TR0322, Network Appliance, 1997. http://tinyurl.com/27ommd.Google ScholarGoogle Scholar
  15. Kleiman, SR. Vnodes: An architecture for multiple file system types in Sun UNIX. In Proc. USENIX Summer 1986 Technical Conference, pages 238--247, 1986.Google ScholarGoogle Scholar
  16. Liskov, B. and R. Rodrigues. Transactional file systems can be fast. In Proc. 11th ACM SIGOPS European Workshop, Sept. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Mann, T., A. Birrell, A. Hisgen, C. Jerian, and G. Swart. A coherent distributed file cache with directory write-behind. ACM Transactions on Computer Systems, 12(2): 123--164, May 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. McKusick, M. K. and G. R. Ganger. Soft updates: A technique for eliminating most synchronous writes in the Fast Filesystem. In Proc. 1999 USENIX Annual Technical Conference, FREENIX Track, pages 1--17, June 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. McKusick, M. K., W. N. Joy, S. J. Leffler, and R. S. Fabry. A fast file system for UNIX. ACM Transactions on Computer Systems, 2 (3): 181--197, Aug. 1984. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Muniswamy-Reddy, K.-K., C. P. Wright, A. Himmer, and E. Zadok. A versatile and user-oriented versioning file system. In Proc. 3rd USENIX Conference on File and Storage Technologies (FAST '04), pages 115--128, Mar. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Nightingale, E.B., P.M. Chen, and J.Flinn. Speculative execution in a distributed file system. In Proc. 20th ACM Symposium on Operating Systems Principles, pages 191--205, Oct. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Nightingale, E.B., K. Veeraraghavan, P.M. Chen, and J. Flinn. Rethink the sync. In Proc. 7th Symposium on Operating Systems Design and Implementation (OSDI '06), pages 1--14, Nov. 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Quinlan, S. and S. Dorward. Venti: a new approach to archival storage. In Proc. 1st USENIX Conference on File and Storage Technologies (FAST '02), pages 89--101, Jan. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Rosenthal, D.S.H. Evolving the Vnode interface. In Proc. USENIX Summer 1990 Technical Conference, pages 107--118, Jan. 1990.Google ScholarGoogle Scholar
  25. Rowe, M. Re: wc atomic rename safety on non-ext3 file systems. Subversion developer mailing list, Mar5 2007. http://svn.haxx.se/dev/archive-2007-03/0064.shtml (retrieved August 2007).Google ScholarGoogle Scholar
  26. Seltzer, MI., GR. Ganger, MK. McKusick, KA. Smith, CAN. Soules, and CA. Stein. Journaling versus soft updates: Asynchronous meta-data protection in file systems. In Proc. 2000 USENIX Annual Technical Conference, pages 71--84, June 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Sivathanu, G., S. Sundararaman, and E. Zadok. Type-safe disks. In Proc. 7th Symposium on Operating Systems Design and Implementation (OSDI '06), pages 15--28, Nov. 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Sivathanu, M., V. Prabhakaran, F. Popovici, T. Denehy, AC. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Semantically-smart disk systems. In Proc. 2nd USENIX Conference on File and Storage Technologies (FAST '03), Mar. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Sivathanu, M., A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, and S. Jha. A logic of file systems. In Proc. 4th USENIX Conference on File and Storage Technologies (FAST '05), pages 1--15, Dec. 2005\natexlaba. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Sivathanu, M., L. N. Bairavasundaram, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Database-aware semantically-smart storage. In Proc. 4th USENIX Conference on File and Storage Technologies (FAST '05), pages 239--252, Dec. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Skinner, G. C. and T. K. Wong. "Stacking" Vnodes: A progress report. In Proc. USENIX Summer 1993 Technical Conference, pages 161--174, June 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Soules, C. A. N., G. R. Goodson, J. D. Strunk, and G. R. Ganger. Metadata efficiency in versioning file systems. In Proc. 2nd USENIX Conference on File and Storage Technologies (FAST '03), pages 43--58, Mar. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Ts'o, T. Re: {evals} ext3 vs reiser with quotas, Dec19 2004. http://linuxmafia.com/faq/Filesystems/reiserfs.html (retrieved August 2007).Google ScholarGoogle Scholar
  34. Tweedie, S. Journaling the Linux ext2fs filesystem. In Proc. 4th Annual LinuxExpo, 1998.Google ScholarGoogle Scholar
  35. Vilayannur, M., PNath, and ASivasubramaniam. Providing tunable consistency for a parallel file store. In Proc. 4th USENIX Conference on File and Storage Technologies (FAST '05), pages 17--30, Dec. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Waychison, M. Re: fallocate support for bitmap-based files. linux-ext4 mailing list, June~29 2007. http://www.mail-archive.com/l[email protected]/msg02382.html (retrieved August 2007).Google ScholarGoogle Scholar
  37. Wright, C. P. Extending ACID Semantics to the File System via ptrace. PhD thesis, Stony Brook University, May 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Wright, C. P., M. C. Martino, and E. Zadok. NCryptfs: A secure and convenient cryptographic file system. In Proc. 2003 USENIX Annual Technical Conference, pages 197--210, June 2003.Google ScholarGoogle Scholar
  39. Wright, C. P., J. Dave, P. Gupta, H. Krishnan, D. P. Quigley, E. Zadok, and M. N. Zubair. Versatility and Unix semantics in namespace unification. ACM Transactions on Storage, Mar. 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Yang, J., P. Twohey, D. Engler, and M. Musuvathni. Using model checking to find serious file system errors. In Proc. 6th Symposium on Operating Systems Design and Implementation (OSDI '04), pages 273--288, Dec. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Yang, J., C. Sar, and D. Engler. eXplode: a lightweight, general system for finding serious storage system errors. In Proc. 7th Symposium on Operating Systems Design and Implementation (OSDI '06), pages 131--146, Nov. 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Zadok, E. and J. Nieh. FiST: A language for stackable file systems. In Proc. 2000 USENIX Annual Technical Conference, pages 55--70, June 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Zadok, E., I. Badulescu, and A. Shender. Extending File Systems Using Stackable Templates. In Proc. 1999 USENIX Annual Technical Conference, pages 57--70, June 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Generalized file system dependencies

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGOPS Operating Systems Review
          ACM SIGOPS Operating Systems Review  Volume 41, Issue 6
          SOSP '07
          December 2007
          363 pages
          ISSN:0163-5980
          DOI:10.1145/1323293
          Issue’s Table of Contents
          • cover image ACM Conferences
            SOSP '07: Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles
            October 2007
            378 pages
            ISBN:9781595935915
            DOI:10.1145/1294261

          Copyright © 2007 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 14 October 2007

          Check for updates

          Qualifiers

          • article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!