skip to main content
research-article
Public Access

Empirical Evaluation and Enhancement of Enterprise Storage System Request Scheduling

Published:27 April 2018Publication History
Skip Abstract Section

Abstract

Since little has been reported in the literature concerning enterprise storage system file-level request scheduling, we do not have enough knowledge about how various scheduling factors affect performance. Moreover, we are in lack of a good understanding on how to enhance request scheduling to adapt to the changing characteristics of workloads and hardware resources. To answer these questions, we first build a request scheduler prototype based on WAFL®, a mainstream file system running on numerous enterprise storage systems worldwide. Next, we use the prototype to quantitatively measure the impact of various scheduling configurations on performance on a NetApp®'s enterprise-class storage system. Several observations have been made. For example, we discover that in order to improve performance, the priority of write requests and non-preempted restarted requests should be boosted in some workloads. Inspired by these observations, we further propose two scheduling enhancement heuristics called SORD (size-oriented request dispatching) and QATS (queue-depth aware time slicing). Finally, we evaluate them by conducting a wide range of experiments using workloads generated by SPC-1 and SFS2014 on both HDD-based and all-flash platforms. Experimental results show that the combination of the two can noticeably reduce average request latency under some workloads.

References

  1. 2015. Bandwidth (Computing). Retrieved Sept. 11, 2015 from https://en.wikipedia.org/wiki/Bandwidth_(computing).Google ScholarGoogle Scholar
  2. 2015. {NetApp FAS8080 EX Datasheet. Retrieved Sept. 11, 2015 from http://www.netapp.com/us/media/ds-3580.pdf.Google ScholarGoogle Scholar
  3. 2015. P320h HHHL PCIe Enterprise SSD. Retrieved Oct. 19, 2015 from https://www.micron.com/∼/media/documents/products/product-flyer/brief_p320h_hhhl.pdf.Google ScholarGoogle Scholar
  4. 2015. SPEC SFS 2014 Benchmark. Retrieved Sept. 11, 2015 from https://www.spec.org/sfs2014/.Google ScholarGoogle Scholar
  5. 2015. ZFS at OpenSolaris community. Retrieved Sept. 11, 2015 from http://opensolaris.org/os/community/zfs/.Google ScholarGoogle Scholar
  6. Parag Agrawal, Daniel Kifer, and Christopher Olston. 2008. Scheduling shared scans of large data files. Proceedings of the VLDB Endowment 1, 1 (2008), 958--969. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Giridhar Appaji Nag Yasa and P. C. Nagesh. 2012. Space savings and design considerations in variable length deduplication. ACM SIGOPS Operating Systems Review 46, 3 (2012), 57--64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Alexandros Batsakis, Randal Burns, Arkady Kanevsky, James Lentini, and Thomas Talpey. 2009. CA-NFS: A congestion-aware network file system. ACM Transactions on Storage (TOS) 5, 4 (2009), 15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ellie Berriman, Paul Feresten, and Shawn Kung. 2015. IDC worldwide quarterly enterprise storage systems tracker.Google ScholarGoogle Scholar
  10. David Boutcher and Abhishek Chandra. 2010. Does virtualization make disk scheduling passé? ACM SIGOPS Operating Systems Review 44, 1 (2010), 20--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. John Bruno, Jose Brustoloni, Eran Gabber, Banu Özden, and Abraham Silberschatz. 1999. Disk scheduling with quality of service guarantees. In Proceedings of the IEEE International Conference on Multimedia Computing and Systems, 1999, Vol. 2. IEEE, 400--405. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Shenze Chen, John A. Stankovic, James F. Kurose, and Don Towsley. 1991. Performance evaluation of two new disk scheduling algorithms for real-time systems. Real-Time Systems 3, 3 (1991), 307--336.Google ScholarGoogle ScholarCross RefCross Ref
  13. Storage Performance Council. 2017. PC Benchmark 1 (SPC-1) Specification - Revision 3.1. http://www.storageperformance.org/specs/SPC1_v310.pdf.Google ScholarGoogle Scholar
  14. Matthew Curtis-Maury, Vinay Devadas, Vania Fang, and Aditya Kulkarni. 2016. To waffinity and beyond: A scalable architecture for incremental parallelization of file system code. In Proceedings of the12th USENIX Symposium on Operating Systems Design and Implementation (OSDI’16). USENIX Association, 419--434. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Scott Dawkins, Kaladhar Voruganti, and John D. Strunk. 2012. Systems research and innovation in data ONTAP. ACM SIGOPS Operating Systems Review 46, 3 (2012), 1--3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. John K. Edwards, Daniel Ellard, Craig Everhart, Robert Fair, Eric Hamilton, Andy Kahn, Arkady Kanevsky, James Lentini, Ashish Prakash, Keith A Smith, and others. 2008. FlexVol: Flexible, efficient file volume virtualization in WAFL. In Proceedings of the USENIX 2008 Annual Technical Conference on Annual Technical Conference. USENIX Association, 129--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. EMC. 2015. EMC Isilon OneFS: A Technical Overview. https://www.emc.com/collateral/hardware/white-papers/h10719-isilon-onefs-technical-overview-wp.pdf.Google ScholarGoogle Scholar
  18. Binny S. Gill and Dharmendra S. Modha. 2005. Wow: Wise ordering for writes-combining spatial and temporal locality in non-volatile caches. In Proceedings of the 4th Conference on USENIX Conference on File and Storage Technologies-Volume 4. USENIX Association, 10--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Anoop Gupta, Andrew Tucker, and Shigeru Urushibara. 1991. The impact of operating system scheduling policies and synchronization methods of performance of parallel applications. In ACM SIGMETRICS Performance Evaluation Review, Vol. 19. ACM, 120--132. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Dave Hitz and others. 1994. File system design for an NFS file server appliance. In USENIX Winter, Vol. 94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Sitaram Iyer and Peter Druschel. 2001. Anticipatory scheduling: A disk scheduling framework to overcome deceptive idleness in synchronous I/O. In ACM SIGOPS Operating Systems Review, Vol. 35. ACM, 117--130. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Jaeho Kim, Yongseok Oh, Eunsam Kim, Jongmoo Choi, Donghee Lee, and Sam H. Noh. 2009. Disk schedulers for solid state drivers. In Proceedings of the 7th ACM International Conference on Embedded Software. ACM, 295--304. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Youngjae Kim, Scott Atchley, Geoffroy R. Vallée, and Galen M. Shipman. 2015. LADS: Optimizing data transfers using layout-aware data scheduling. In Proceedings of the 13th USENIX Conference on File and Storage Technologies. USENIX Association, 67--80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Andrew W. Leung, Shankar Pasupathy, Garth R. Goodson, and Ethan L. Miller. 2008. Measurement and analysis of large-scale network file system workloads. In Proceedings of the USENIX Annual Technical Conference, Vol. 1. 5--2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Peter Macko, Margo I. Seltzer, and Keith A. Smith. 2010. Tracking back references in a write-anywhere file system. In Proceedings of FAST. 15--28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Marshall Kirk McKusick and George V. Neville-Neil. 2004. Thread scheduling in FreeBSD 5.2. Queue 2, 7 (2004), 58--64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Hugo Patterson, Stephen Manley, Mike Federwisch, Dave Hitz, Steve Kleiman, and Shane Owara. 2002. SnapMirror®: File system based asynchronous mirroring for disaster recovery. In Proceedings of the 1st USENIX Conference on File and Storage Technologies. USENIX Association, 9--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Anna Povzner, Tim Kaldewey, Scott Brandt, Richard Golding, Theodore M. Wong, and Carlos Maltzahn. 2008. Efficient guaranteed disk request scheduling with fahrrad. In ACM SIGOPS Operating Systems Review, Vol. 42. ACM, 13--25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. AL Reddy and Jim Wyllie. 1993. Disk scheduling in a multimedia I/O system. In Proceedings of the 1st ACM International Conference on Multimedia. ACM, 225--233. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Ohad Rodeh. 2008. B-trees, shadowing, and clones. ACM Transactions on Storage (TOS) 3, 4 (2008), 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Y. Rompogiannakis, Guido Nerjes, Peter Muth, Michael Paterakis, Peter Triantafillou, and Gerhard Weikum. 1998. Disk scheduling for mixed-media workloads in a multimedia server. In Proceedings of the 6th ACM International Conference on Multimedia. ACM, 297--302. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Prashant J. Shenoy and Harrick M. Vin. 1998. Cello: A disk scheduling framework for next generation operating systems. In ACM SIGMETRICS Performance Evaluation Review, Vol. 26. ACM, 44--55. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Alexander Thomasian. 2011. Survey and analysis of disk scheduling methods. ACM SIGARCH Computer Architecture News 39, 2 (2011), 8--25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Avishay Traeger, Erez Zadok, Nikolai Joukov, and Charles P. Wright. 2008. A nine year study of file system and storage benchmarking. ACM Transactions on Storage (TOS) 4, 2 (2008), 5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Yuehai Xu and Song Jiang. 2011. A scheduling framework that makes any disk schedulers non-work-conserving solely based on request characteristics. In Proceedings of FAST. 119--132. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Chang-Woo Yang, Adam Wierman, Sanjay Shakkottai, and Mor Harchol-Balter. 2006. Tail asymptotics for policies favoring short jobs in a many-flows regime. ACM SIGMETRICS Performance Evaluation Review 34, 1 (2006), 97--108. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, and Ion Stoica. 2010. Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling. In Proceedings of the 5th European Conference on Computer Systems. ACM, 265--278. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Empirical Evaluation and Enhancement of Enterprise Storage System Request Scheduling

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Storage
      ACM Transactions on Storage  Volume 14, Issue 2
      May 2018
      210 pages
      ISSN:1553-3077
      EISSN:1553-3093
      DOI:10.1145/3208078
      • Editor:
      • Sam H. Noh
      Issue’s Table of Contents

      Copyright © 2018 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 27 April 2018
      • Accepted: 1 February 2018
      • Revised: 1 December 2017
      • Received: 1 April 2017
      Published in tos Volume 14, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed
    • Article Metrics

      • Downloads (Last 12 months)25
      • Downloads (Last 6 weeks)2

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!