skip to main content
research-article

FluidSMR: Adaptive Management for Hybrid SMR Drives

Published:15 October 2021Publication History
Skip Abstract Section

Abstract

Hybrid Shingled Magnetic Recording (H-SMR) drives are the most recently developed SMR drives, which allow dynamic conversion of the recording format between Conventional Magnetic Recording (CMR) and SMR on a single disk drive. We identify the unique opportunities of H-SMR drives to manage the tradeoffs between performance and capacity, including the possibility of adjusting the SMR area capacity based on storage usage and the flexibility of dynamic data swapping between the CMR area and SMR area.

We design and implement FluidSMR, an adaptive management scheme for hybrid SMR Drives, to fully utilize H-SMR drives under different workloads and capacity usages. FluidSMR has a two-phase allocation scheme to support a growing usage of the H-SMR drive. The scheme can intelligently determine the sizes of the CMR and the SMR space in an H-SMR drive based on the dynamic changing of workloads. Moreover, FluidSMR uses a cache in the CMR region, managed by a proposed loop-back log policy, to reduce the overhead of updates to the SMR region.

Evaluations using enterprise traces demonstrate that FluidSMR outperforms baseline schemes in various workloads by decreasing the average I/O latency and effectively reducing/controlling the performance impact of the format conversion between CMR and SMR.

REFERENCES

  1. [1] Aghayev Abutalib, Shafaei Mansour, and Desnoyers Peter. 2015. Skylight—A window on shingled disk operation. ACM Trans. Stor. 11, 4, Article 16 (Oct. 2015), 28 pages. https://doi.org/10.1145/2821511Google ScholarGoogle Scholar
  2. [2] Amer Ahmed, Holliday JoAnne, Long Darrell D. E., Miller Ethan L, Pâris Jehan-François, and Schwarz Thomas. 2011. Data management and layout for shingled magnetic recording. IEEE Trans. Magn. 47, 10 (2011), 36913697.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Amer Ahmed, Long Darrell D. E., Miller Ethan L., Paris J.-F., and Schwarz S. J. T.. 2010. Design issues for a shingled write disk system. In Proceedings of the IEEE Conference on Mass Storage Systems and Technologies (MSST'10).Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Armstrong Timothy G., Ponnekanti Vamsi, Borthakur Dhruba, and Callaghan Mark. 2013. LinkBench: A database benchmark based on the Facebook social graph. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data. 11851196.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Bloom Burton H.. 1970. Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13, 7 (1970), 422426.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Boyle Bill and Stevens Curtis E.. [n.d.]. Realms API. Retrieved from http://www.t10.org/cgi-bin/ac.pl?t=d&f=17-158r1.pdf.Google ScholarGoogle Scholar
  7. [7] Bucy John S., Schindler Jiri, Schlosser Steven W., and Ganger Gregory R.. 2008. The disksim Simulation Environment Version 4.0 Reference Manual (cmu-pdl-08-101). Parallel Data Laboratory (2008), 26.Google ScholarGoogle Scholar
  8. [8] Cao Zhichao, Dong Siying, Vemuri Sagar, and Du David H. C.. 2020. Characterizing, modeling, and benchmarking RocksDB key-value workloads at Facebook. In Proceedings of the 18th USENIX Conference on File and Storage Technologies (FAST'20). 209223.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Cao Zhichao, Wen Hao, Ge Xiongzi, Ma Jingwei, Diehl Jim, and Du David H. C.. 2019. TDDFS: A tier-aware data deduplication-based file system. ACM Trans. Stor. 15, 1 (2019), 126.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Cassuto Yuval, Sanvido Marco A. A., Guyot Cyril, Hall David R., and Bandic Zvonimir Z.. 2010. Indirection systems for shingled-recording disk drives. In Proceedings of the IEEE Conference on Mass Storage Systems and Technologies (MSST'10).Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Chen Feng, Koufaty David A., and Zhang Xiaodong. 2011. Hystor: Making the best use of solid state drives in high performance storage systems. In Proceedings of the International Conference on Supercomputing. ACM, 2232.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Berg Mark De, Kreveld Marc Van, Overmars Mark, and Schwarzkopf Otfried. 1997. Computational geometry. In Computational Geometry. Springer, 117.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Dropbox. 2018. Extending Magic Pocket Innovation with the First Petabyte Scale SMR Drive Deployment. Retrieved November 2019 from https://blogs.dropbox.com/tech/2018/06/extending-magic-pocket-innovation-with-the-first-petabyte-scale-smr-drive-deployment/.Google ScholarGoogle Scholar
  14. [14] Fan Ziqi, Wu Fenggang, Diehl Jim, Du David H. C., and Voigt Doug. 2018. CDBB: An NVRAM-based burst buffer coordination system for parallel file systems. In Proceedings of the High Performance Computing Symposium. Society for Computer Simulation International, 1.Google ScholarGoogle Scholar
  15. [15] Feldman Timothy. [n.d.]. Flex Overview. Retrieved from http://t13.org/Documents/UploadedDocuments/docs2018/f17156r0-Flex_Overview.pdf.Google ScholarGoogle Scholar
  16. [16] Feldman Tim. 2018. Flex dynamic recording. USENIX ;login: 43, 1 (2018).Google ScholarGoogle Scholar
  17. [17] Gao Kaizhong, Zhu Wenzhong, and Gage Edward. 2016. Write management for interlaced magnetic recording devices. US Patent 9,508,362.Google ScholarGoogle Scholar
  18. [18] Gao Kaizhong, Zhu Wenzhong, and Gage Edward. 2017. Interlaced magnetic recording. US Patent 9,728,206.Google ScholarGoogle Scholar
  19. [19] Ge Xiongzi, Xie Xuchao, Du David H. C., Ganesan Pradeep, and Hahn Dennis. 2018. Chewanalyzer: Workload-aware data management across differentiated storage pools. In Proceedings of the IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS'18). IEEE, 94101.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Google. [n.d.]. Dynamic Hybrid-SMR: An OCP Proposal to Improve Data Center Disk Drives. Retrieved from https://blog.google/products/google-cloud/dynamic-hybrid-smr-ocp-proposal-improve-data-center-disk-drives/.Google ScholarGoogle Scholar
  21. [21] Guerra Jorge, Pucha Himabindu, Glider Joseph S., Belluomini Wendy, and Rangaswami Raju. 2011. Cost effective storage using extent based dynamic tiering. In Proceedings of the USENIX Conference on File and Storage TechnologiesFAST, Vol. 11. 2020.Google ScholarGoogle Scholar
  22. [22] Hajkazemi Mohammad Hossein, Kulkarni Ajay Narayan, Desnoyers Peter, and Feldman Timothy R.. 2019. Track-based translation layers for interlaced magnetic recording. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC'19). 821832.Google ScholarGoogle Scholar
  23. [23] Hall David, Marcos John H., and Coker Jonathan D.. 2012. Data handling algorithms for autonomous shingled magnetic recording hdds. IEEE Trans. Magn. 48, 5 (2012), 17771781.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] He Weiping and Du David H. C.. 2014. Novel address mappings for shingled write disks. In Proceedings of the HotStorage'14.Google ScholarGoogle Scholar
  25. [25] He Weiping and Du David H. C.. 2017. SMaRT: An approach to shingled magnetic recording translation. In Proceedings of the 15th USENIX Conference on File and Storage Technologies (FAST'17).Google ScholarGoogle Scholar
  26. [26] Committee INCITS T10 Technical. 2015. Information Technology—Zoned Block Commands (ZBC). Retrieved from http://www.t10.org/drafts.htm.Google ScholarGoogle Scholar
  27. [27] Committee INCITS T13 Technical. [n.d.]. Zoned-device ATA Command Set (ZAC) working draft.Google ScholarGoogle Scholar
  28. [28] Jin Chao, Xi Wei-Ya, Ching Zhi-Yong, Huo Feng, and Lim Chun-Teck. 2014. HiSMRfs: A high performance file system for shingled storage array. In Proceedings of the 30th Symposium on Mass Storage Systems and Technologies (MSST'14). 16. https://doi.org/10.1109/MSST.2014.6855539Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Kadekodi Saurabh, Pimpale Swapnil, and Gibson Garth A.. 2015. Caveat-scriptor: Write anywhere shingled disks. In Proceedings of the ACM Workshop on Hot Topics in Storage and File Systems (HotStorage'15).Google ScholarGoogle Scholar
  30. [30] Kim Youngjae, Gupta Aayush, Urgaonkar Bhuvan, Berman Piotr, and Sivasubramaniam Anand. 2011. HybridStore: A cost-efficient, high-performance storage system combining SSDs and HDDs. In Proceedings of the IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems. IEEE, 227236.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Lim Sang-Phil, Lee Sang-Won, and Moon Bongki. 2010. FASTer FTL for enterprise-class flash memory SSDs. In Proceedings of the International Workshop on Storage Network Architecture and Parallel I/Os (SNAPI'10). IEEE, 312.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Lin Chung-I., Park Dongchul, He Weiping, and Du David H. C.. 2012. H-SWD: Incorporating hot data identification into shingled write disks. In Proceedings of the 20th IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS'12).Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Macko Peter, Ge Xiongzi, Jr John Haskins, Kelley James, Slik David, Smith Keith A., and Smith Maxim G.. 2017. SMORE: A Cold Data Object Store for SMR Drives (Extended Version). arXiv:1705.09701. Retrieved from https://arxiv.org/abs/1705.09701.Google ScholarGoogle Scholar
  34. [34] Macko Peter, Ge Xiongzi, Kelley J., Slik D., et al. 2017. SMORE: A cold data object store for SMR drives. In Proceedings of the 33rd International Conference on Massive Storage Systems and Technology (MSST'17).Google ScholarGoogle Scholar
  35. [35] Manzanares Adam, Watkins Noah, Guyot Cyril, LeMoal Damien, Maltzahn Carlos, and Bandic Zvonimr. 2016. ZEA, A data management approach for SMR. In Proceedings of the 8th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage'16).Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Mao Yu, Wan Jiguang, Zhu Yifeng, and Xie Changsheng. 2014. A new parity-based migration method to expand RAID-5. IEEE Trans. Parallel Distrib. Syst. 25, 8 (2014), 19451954.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Narayanan Dushyanth, Donnelly Austin, and Rowstron Antony. 2008. Write off-loading: Practical power management for enterprise storage. ACM Trans. Stor. 4, 3 (2008), 10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Park Dongchul, Lin Chung-I, and Du David H. C.. 2012. H-SWD: A novel shingled write disk scheme based on hot and cold data identification. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST'12).Google ScholarGoogle Scholar
  39. [39] Pitchumani Rekha, Hughes James, and Miller Ethan L.. 2015. SMRDB: Key-value data store for shingled magnetic recording disks. In Proceedings of the 8th ACM International Systems and Storage Conference. ACM, 18.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Rodrigues Goldwyn. 2014. Flushing out pdflush. Retrieved from https://lwn.net/Articles/326552/.Google ScholarGoogle Scholar
  41. [41] Ruemmler Chris and Wilkes John. 1994. An introduction to disk drive modeling. Computer 27, 3 (1994), 1728.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Technology Seagate. [n.d.]. New Flex Dynamic Recording Method Redefines the Data Center Hard Drive. Retrieved from https://blog.seagate.com/intelligent/new-flex-dynamic-recording-method-redefines-data-center-hard-drive/.Google ScholarGoogle Scholar
  43. [43] Strunk John D.. 2012. Hybrid aggregates: Combining SSDs and HDDs in a single storage pool. ACM SIGOPS Operat. Syst. Rev. 46, 3 (2012), 5056.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Feldman Timothy. 2017. Flex Device Interface. Retrieved March 2020 from http://t13.org/Documents/Uploaded Documents/docs2018/f18101r0-Flex_Device_Interface.pdf.Google ScholarGoogle Scholar
  45. [45] Wang Chunling, Wang Dandan, Chai Yupeng, Wang Chuanwen, and Sun Diansen. 2017. Larger, cheaper, but faster: SSD-SMR hybrid storage boosted by a new SMR-oriented cache framework. In Proceedings of the IEEE Symposium Mass Storage Systems and Technology (MSST'17).Google ScholarGoogle Scholar
  46. [46] Digital Western. [n.d.]. Dynamic Hybrid SMR. Retrieved from https://blog.westerndigital.com/dynamic-hybrid-smr/.Google ScholarGoogle Scholar
  47. [47] Wilkes John, Golding Richard, Staelin Carl, and Sullivan Tim. 1996. The HP AutoRAID hierarchical storage system. ACM Trans. Comput. Syst. 14, 1 (1996), 108136.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Wu Chentao and He Xubin. 2012. GSR: A global stripe-based redistribution approach to accelerate RAID-5 scaling. In Proceedings of the 41st International Conference on Parallel Processing. IEEE, 460469.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Wu Chentao, He Xubin, Han Jizhong, Tan Huailiang, and Xie Changsheng. 2012. SDM: A stripe-based data migration scheme to improve the scalability of RAID-6. In Proceedings of the IEEE International Conference on Cluster Computing. IEEE, 284292.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Wu Chentao, He Xubin, Li Jie, and Guo Minyi. 2015. Code 5-6: An efficient MDS array coding scheme to accelerate online RAID level migration. In Proceedings of the 44th International Conference on Parallel Processing. IEEE, 450459.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Wu Fenggang, Fan Ziqi, Yang Ming-Chang, Zhang Baoquan, Ge Xiongzi, and Du David H. C.. 2017. Performance evaluation of host aware shingled magnetic recording (HA-SMR) drives. IEEE Trans. Comput. 66, 11 (2017), 19321945.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. [52] Wu Fenggang, Li Bingzhe, Cao Zhichao, Zhang Baoquan, Yang Ming-Hong, Wen Hao, and Du David H. C.. 2019. ZoneAlloy: Elastic data and space management for hybrid SMR drives. In Proceedings of the 11th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage'19).Google ScholarGoogle Scholar
  53. [53] Wu Fenggang, Li Bingzhe, Zhang Baoquan, Cao Zhichao, Diehl Jim, Wen Hao, and Du David HC. 2020. Tracklace: Data management for interlaced magnetic recording. IEEE Transactions on Computers 70, 3 (2020), 347358.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. [54] Wu Fenggang, Yang Ming-Chang, Fan Ziqi, Zhang Baoquan, Ge Xiongzi, and Du David H.C.. 2016. Evaluating host aware SMR drives. In Proceedings of the 8th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage'16).Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. [55] Wu Fenggang, Zhang Baoquan, Cao Zhichao, Wen Hao, Li Bingzhe, Diehl Jim, Wang Guohua, and Du David H. C.. 2018. Data management design for interlaced magnetic recording. In Proceedings of the 10th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage'18).Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. [56] Xiao Wenjian, Dong Huanqing, Ma Liuying, Liu Zhenjun, and Zhang Qiang. 2016. HS-BAS: A hybrid storage system based on band awareness of Shingled Write Disk. In Proceedings of the IEEE 34th International Conference on Computer Design (ICCD'16). IEEE, 6471.Google ScholarGoogle ScholarCross RefCross Ref
  57. [57] Xie Xuchao, Xiao Liquan, and Du David H. C.. 2019. ZoneTier: A zone-based storage tiering and caching co-design to integrate SSDs with SMR drives. ACM Trans. Stor. 15, 3 (2019), 19.Google ScholarGoogle Scholar
  58. [58] Yang Ming-Chang, Chang Yuan-Hao, Wu Fenggang, Kuo Tei-Wei, and Du David H. C.. 2017. Virtual persistent cache: Remedy the long latency behavior of host-aware shingled magnetic recording drives. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD'17). IEEE, 1724.Google ScholarGoogle ScholarCross RefCross Ref
  59. [59] Yang Ming-Chang, Chang Yuan-Hao, Wu Fenggang, Kuo Tei-Wei, and Du David H. C.. 2018. On improving the write responsiveness for host-aware SMR drives. IEEE Trans. Comput. 68, 1 (2018), 111124.Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. [60] Yao Ting, Wan Jiguang, Huang Ping, Zhang Yiwen, Liu Zhiwen, Xie Changsheng, and He Xubin. 2019. GearDB: A GC-free key-value store on HM-SMR drives with gear compaction. In Proceedings of the 17th USENIX Conference on File and Storage Technologies (FAST'19). 159171.Google ScholarGoogle Scholar
  61. [61] Zhang B., Yang M., Xie X., and Du D. H. C.. 2020. Idler: I/O workload controlling for better responsiveness on host-aware shingled magnetic recording drives. IEEE Trans. Comput. (2020), 11. https://doi.org/10.1109/TC.2020.2966194Google ScholarGoogle Scholar
  62. [62] Zhang Guangyan, Shu Jiwu, Xue Wei, and Zheng Weimin. 2007. SLAS: An efficient approach to scaling round-robin striped volumes. ACM Trans. Stor. 3, 1 (2007), 3–es.Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. [63] Zhang Guangyan, Zheng Weiman, and Shu Jiwu. 2009. ALV: A new data redistribution approach to RAID-5 scaling. IEEE Trans. Comput. 59, 3 (2009), 345357.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. [64] Zheng Weimin and Zhang Guangyan. 2011. Fastscale: Accelerate raid scaling by minimizing data migration. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST'11). 149161.Google ScholarGoogle Scholar

Index Terms

  1. FluidSMR: Adaptive Management for Hybrid SMR Drives

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Storage
          ACM Transactions on Storage  Volume 17, Issue 4
          November 2021
          201 pages
          ISSN:1553-3077
          EISSN:1553-3093
          DOI:10.1145/3487989
          • Editor:
          • Sam H. Noh
          Issue’s Table of Contents

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 15 October 2021
          • Accepted: 1 May 2021
          • Revised: 1 February 2021
          • Received: 1 May 2020
          Published in tos Volume 17, Issue 4

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        View Full Text

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!