skip to main content
research-article

Efficient Free Space Reclamation in WAFL

Published:28 September 2017Publication History
Skip Abstract Section

Abstract

NetApp®WAFL® is a transactional file system that uses the copy-on-write mechanism to support fast write performance and efficient snapshot creation. However, copy-on-write increases the demand on the file system to find free blocks quickly, which makes rapid free space reclamation essential. Inability to find free blocks quickly may impede allocations for incoming writes. Efficiency is also important, because the task of reclaiming free space may consume CPU and other resources at the expense of client operations. In this article, we describe the evolution (over more than a decade) of the WAFL algorithms and data structures for reclaiming space with minimal impact to the overall performance of the storage appliance.

References

  1. Michael A. Bender, Martin Farach-Colton, Jeremy T. Fineman, Yonatan R. Fogel, Bradley C. Kuszmaul, and Jelani Nelson. 2007. Cache-oblivious streaming b-trees. In Proceedings of ACM Symposium on Parallel Algorithms and Architectures. 81--92. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Jeff Bonwick. 2007. Space Maps. Retrieved from https://blogs.oracle.com/bonwick/en/entry/space_maps.Google ScholarGoogle Scholar
  3. Mingming Cao, Jose R. Santos, and Andreas Dilger. 2008. Ext4 block and inode allocator improvements. In Linux Symposium. 263.Google ScholarGoogle Scholar
  4. Peter Corbett, Bob English, Atul Goel, Tomislav Grcanac, Steven Kleiman, James Leong, and Sunitha Sankar. 2004. Row-diagonal parity for double disk failure correction. Proceedings of USENIX Conference on File and Storage Technologies (FAST’04).Google ScholarGoogle Scholar
  5. Standard Performance Evaluation Corporation. SPEC SFS 2014. Retrieved from https://www.spec.org/sfs2014/.Google ScholarGoogle Scholar
  6. Storage Performance Council. Storage Performance Council-1 Benchmark. Retrieved from www.storageperformance.org/results/#spc1_overview.Google ScholarGoogle Scholar
  7. Matthew Curtis-Maury, Vinay Devadas, Vania Fang, and Aditya Kulkarni. 2016. To waffinity and beyond: A scalable architecture for incremental parallelization of file system code. In Proceedings of Symposium on Operating Systems Design and Implementation (OSDI’16).Google ScholarGoogle Scholar
  8. Matthew Curtis-Maury, Ram Kesavan, and Mrinal K. Bhattacharjee. 2017. Scalable write allocation in the WAFL file system. In Proceedings of International Conference on Parallel Processing (ICPP’17). Google ScholarGoogle ScholarCross RefCross Ref
  9. Peter R. Denz, Matthew Curtis-Maury, and Vinay Devadas. 2016. Think global, act local: A buffer cache design for global ordering and parallel processing in the WAFL file system. In Proceeding of 45th International Conference on Parallel Processing (ICPP’16). Google ScholarGoogle ScholarCross RefCross Ref
  10. John K. Edwards, Daniel Ellard, Craig Everhart, Robert Fair, Eric Hamilton, Andy Kahn, Arkady Kanevsky, James Lentini, Ashish Prakash, Keith A. Smith, and Edward Zayas. 2008. FlexVol: Flexible, efficient file volume virtualization in WAFL. In Proceedings of the 2008 USENIX Annual Technical Conference. 129--142.Google ScholarGoogle Scholar
  11. Travis R. Grusecki. 2012. Improving block sharing in the Write Anywhere File Layout file system. http://hdl.handle.net/1721.1/76818.Google ScholarGoogle Scholar
  12. Christopher Hertel. 2003. Implementing CIFS: The Common Internet File System. Prentice Hall Professional Technical Reference.Google ScholarGoogle Scholar
  13. Dave Hitz, James Lau, and Michael Malcolm. 1994. File system design for an NFS file server appliance. In Proceedings of USENIX Winter 1994 Technical Conference. 235--246.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. NetApp Inc. 2010. Data ONTAP 8. Retrieved from http://www.netapp.com/us/products/platform-os/ontap/.Google ScholarGoogle Scholar
  15. Ram Kesavan, Rohit Singh, Travis Grusecki, and Yuvraj Patel. 2017. Algorithms and data structures for efficient free space reclamation in WAFL. In 15th USENIX Conference on File and Storage Technologies (FAST’17), Geoff Kuenning and Carl A. Waldspurger (Eds.). USENIX Association, 1--14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Ram Kesavan, Sriram Venketaraman, Mohit Gupta, and Subramaniam Periyagaram. 2014. Systems and methods for instantaneous cloning. Patent US8812450 (2014). Retrieved from https://www.google.com/patents/US8812450.Google ScholarGoogle Scholar
  17. Harendra Kumar, Yuvraj Patel, Ram Kesavan, and Sumith Makam. 2017. High performance metadata integrity protection in the WAFL copy-on-write file system. In Proceedings of Conference on File and Storage Technologies (FAST’17).Google ScholarGoogle Scholar
  18. Avantika Mathur, Mingming Cao, and Andreas Dilger. 2007. Ext4: The Next Generation of the Ext3 File System. Usenix Association.Google ScholarGoogle Scholar
  19. Marshall K. McKusick, William N. Joy, Samuel J. Leffler, and Robert S. Fabry. 1984. A fast file system for UNIX. ACM Trans. Comput. Syst. 2, 3 (Aug. 1984), 181--197. DOI:http://dx.doi.org/10.1145/989.990 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Marshall Kirk McKusick, Willian N. Joy, Samuel J. Leffler, and Robert S. Fabry. 1986. Fsck - The UNIX File System Check Program. Unix System Manager’s Manual - 4.3 BSD Virtual VAX-11 Version.Google ScholarGoogle Scholar
  21. Sun Microsystems. ZFS at OpenSolaris community. Retrieved from http://opensolaris.org/os/community/zfs/.Google ScholarGoogle Scholar
  22. Hugo Patterson, Stephen Manley, Mike Federwisch, Dave Hitz, Steve Kleiman, and Shane Owara. 2002. SnapMirror: File system based asychronous mirroring for disaster recovery. Proceedings of USENIX Conference on File and Storage Technologies (FAST’02).Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Peter M. Ridge and David Deming. 1995. The Book of SCSI. No Starch Press, San Francisco, CA.Google ScholarGoogle Scholar
  24. Ohad Rodeh, Josef Bacik, and Chris Mason. 2013. BTRFS: The Linux b-tree filesystem. Trans. Storage 9, 3, Article 9 (Aug. 2013), 32 pages. DOI:http://dx.doi.org/10.1145/2501620.2501623 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Mendel Rosenblum and John K. Ousterhout. 1992. The design and implementation of a log-structured file system. ACM Trans. Comput. Syst. 10 (1992), 1--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Russel Sandberg, David Goldberg, Steve Kleiman, Dan Walsh, and Bob Lyon. 1985. Design and Implementation of the Sun Network File System. In Proceedings of the Summer 1985 USENIX Technical Conference, 19–130.Google ScholarGoogle Scholar
  27. Ananthan Subramanian, Rimas Svarcas, Ram Kesavan, Srishylam Simharajan, and John Frederick DeStefano. 2017. Incremental transfer with unused block reclamation. Patent US20170031772. https://www.google.com/patents/US20170031772.Google ScholarGoogle Scholar
  28. Adam Sweeney, Doug Doucette, Wei Hu, Curtis Anderson, Mike Nishimoto, and Geoff Peck. 1996. Scalability in the XFS file system. In USENIX Annual Technical Conference.Google ScholarGoogle Scholar
  29. Ralph H. Thornburgh and Barry Schoenborn. 2000. Storage Area Networks. Prentice Hall PTR, Upper Saddle River, NJ.Google ScholarGoogle Scholar
  30. Stephen C. Tweedie. 1998. Journaling the linux ext2fs filesystem. In Proceedings of the 4th Annual Linux Expo (LinuxExpo’98).Google ScholarGoogle Scholar
  31. David D. Wright. 2014. Data deletion in a distributed data storage system. Patent US8819208. Retrieved from www.google.com/patents/US8819208.Google ScholarGoogle Scholar

Index Terms

  1. Efficient Free Space Reclamation in WAFL

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!