Abstract
This article presents a design framework aiming to reduce mass data storage cost in data centers. Its underlying principle is simple: Assume one may noticeably reduce the HDD manufacturing cost by significantly (i.e., at least several orders of magnitude) relaxing raw HDD reliability, which ensures the eventual data storage integrity via low-cost system-level redundancy. This is called system-assisted HDD bit cost reduction. To better utilize both capacity and random IOPS of HDDs, it is desirable to mix data with complementary requirements on capacity and random IOPS in each HDD. Nevertheless, different capacity and random IOPS requirements may demand different raw HDD reliability vs. bit cost trade-offs and hence different forms of system-assisted bit cost reduction. This article presents a software-centric design framework to realize data-adaptive system-assisted bit cost reduction for data center HDDs. Implementation is solely handled by the filesystem and demands only minor change of the error correction coding (ECC) module inside HDDs. Hence, it is completely transparent to all the other components in the software stack (e.g., applications, OS kernel, and drivers) and keeps fundamental HDD design practice (e.g., firmware, media, head, and servo) intact. We carried out analysis and experiments to evaluate its implementation feasibility and effectiveness. We integrated the design techniques into ext4 to further quantitatively measure its impact on system speed performance.
- HBase. 2018. https://hbase.apache.org/.Google Scholar
- HiBench 3.0. 2018. https://github.com/intel-hadoop/HiBench/releases.Google Scholar
- Private Communication with Engineers of a Major HDD Manufacturer. 2018.Google Scholar
- YCSB. 2018. https://github.com/brianfrankcooper/YCSB.Google Scholar
- Lakshmi N. Bairavasundaram, Garth R. Goodson, Shankar Pasupathy, and Jiri Schindler. 2007. An analysis of latent sector errors in disk drives. In ACM SIGMETRICS Perform. Eval. Rev., Vol. 35. ACM, 289--300. Google Scholar
Digital Library
- Stephen Bates. 2013. Using rate-adaptive LDPC codes to maximize the capacity of SSDs. In Proceedings of the Flash Memory Summit. 1--12.Google Scholar
- M. Blaum, J. Brady, J. Bruck, and J. Menon. 1995. EVENODD: An efficient scheme for tolerating double disk failures in RAID architectures. IEEE Trans. Comput. 44, 2 (Feb. 1995), 192--202. Google Scholar
Digital Library
- E. Brewer, L. Ying, L. Greenfield, Robert Cypher, and T. T’so. 2016. Disks for Data Centers. Technical Report. Google. https://ai.google/research/pubs/pub44830.Google Scholar
- P. Chen, E. Lee, G. Gibson, R. Katz, and D. Patterson. 1994. RAID: High-performance, reliable secondary storage. Comput. Surveys 26, 2 (June 1994), 145--185. Google Scholar
Digital Library
- Carlo Curino, Evan P. C. Jones, Samuel Madden, and Hari Balakrishnan. 2011. Workload-aware database monitoring and consolidation. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM, 313--324. Google Scholar
Digital Library
- E. Daniel, C. Mee, and M. Clark. 1999. Magnetic Recording: The First 100 Years. John Wiley 8 Sons.Google Scholar
- A. Dimakis, P. Godfrey, Y. Wu, M. Wainwright, and K. Ramchandran. 2010. Network coding for distributed storage systems. IEEE Trans. Inform. Theory 56, 9 (2010), 4539--4551. Google Scholar
Digital Library
- Timothy A. Ferris and Robert P. Ryan. 2015. Disk drive charging capacitor using motor supply voltage during power failure. US Patent 9,093,105.Google Scholar
- Gregory R. Ganger, John D. Strunk, and Andrew J. Klosterman. 2003. Self-* Storage: Brick-based Storage with Automated Administration. Technical Report. Carnegie Mellon University School of Computer Science, Pittsburgh, PA.Google Scholar
- K. Greenan, X. Li, and J. Wylie. 2010. Flat XOR-based erasure codes in storage systems: Constructions, efficient recovery, and tradeoffs. In Proceedings of the 26th IEEE Symposium on Mass Storage Systems and Technologies. 1--14. Google Scholar
Digital Library
- C. Huang, H. Simitci, Y. Xu, A. Ogus, B. Calder, P. Gopalan, J. Li, and S. Yekhanin. 2012. Erasure coding in windows azure storage. In Proceedings of the USENIX Annual Technical Conference. 15--26. Google Scholar
Digital Library
- A. R. Krishnan, R. Radhakrishnan, B. Vasic, A. Kavcic, W. Ryan, and F. Erden. 2009. 2-D magnetic recording: Read channel modeling and detection. IEEE Trans. Magnetics 45, 10 (Oct. 2009), 3830--3836.Google Scholar
- Tom M. Kroeger and Darrell D. E. Long. 1999. The case for efficient file access pattern modeling. In Proceedings of the 7th Workshop on Hot Topics in Operating Systems. IEEE, 14--19. Google Scholar
Digital Library
- E. Kurtas and B. Vasic. 2005. Coding and Signal Processing for Magnetic Recording Systems. CRC Press.Google Scholar
- Y. Li, H. Wang, X. Zhang, N. Zheng, S. Dahandeh, and T. Zhang. 2017. Facilitating magnetic recording technology scaling for data center hard disk drives through filesystem-level transparent local erasure coding. In Proceedings of the USENIX Conference on File and Storage Technologies. 135--148. Google Scholar
Digital Library
- F. Lim, B. Wilson, and R. Wood. 2010. Analysis of shingle-write readback using magnetic-force microscopy. IEEE Trans. Magnetics 46, 6 (June 2010), 1548--1551.Google Scholar
Cross Ref
- S. Lin and D. J. Costello. 2004. Error Control Coding: Fundamentals and Applications (2nd Ed.). Prentice Hall. Google Scholar
Digital Library
- Jaishankar Moothedath Menon and Krishnakumar Surugucchi. 2014. Method to protect data on a disk drive from uncorrectable media errors. US Patent 8,645,622.Google Scholar
- Michael Mesnier, Eno Thereska, Gregory R. Ganger, Daniel Ellard, and Margo Seltzer. 2004. File classification in self-* storage systems. In Proceedings of the International Conference on Autonomic Computing. IEEE, 44--51. Google Scholar
Digital Library
- K. Miura, E. Yamamoto, H. Aoi, and H. Muraoka. 2009. Estimation of maximum track density in shingles writing. IEEE Trans. Magnetics 45, 10 (Oct. 2009), 3722--3725.Google Scholar
Cross Ref
- A. Moser, K. Takano, D. Margulies, M. Albrecht, Y. Sonobe, Y. Ikeda, S. Sun, and E. Fullerton. 2002. Magnetic recording: Advancing into the future. J. Phys. D: Appl. Phys. 35, 19 (2002), R157.Google Scholar
Cross Ref
- D. Patterson, G. Gibson, and R. Katz. 1988. A case for redundant arrays of inexpensive disks (RAID). In Proceedings of the ACM SIGMOD International Conference on Management of Data. 109--116. Google Scholar
Digital Library
- A. Pavlo, E. Paulson, A. Rasin, D. Abadi, D. DeWitt, S. Madden, and M. Stonebraker. 2009. A comparison of approaches to large-scale data analysis. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 165--178. Google Scholar
Digital Library
- V. Prabhakaran, L. Bairavasundaram, N. Agrawal, H. Gunawi, A. Arpaci-Dusseau, and R. Arpaci-Dusseau. 2005. IRON file systems. In Proceedings of the ACM Symposium on Operating Systems Principles. 206--220. Google Scholar
Digital Library
- M. Sathiamoorthy, M. Asteris, D. Papailiopoulos, A. Dimakis, R. Vadali, S. Chen, and D. Borthakur. 2013. Xoring elephants: Novel erasure codes for big data. In Proc. VLDB Endow., Vol. 6. 325--336. Google Scholar
Digital Library
- Bianca Schroeder, Sotirios Damouras, and Phillipa Gill. 2010. Understanding latent sector errors and how to protect against them. ACM Trans. Storage 6, 3 (2010), 9. Google Scholar
Digital Library
- M. A. Seigler, W. A. Challener, E. Gage, N. Gokemeijer, G. Ju, B. Lu, K. Pelhos, C. Peng, R. E. Rottmayer, X. Yang, H. Zhou, and T. Rausch. 2008. Integrated head assisted magnetic recording head: Design and recording demonstration. IEEE Trans. Magnetics 44, 1 (Jan. 2008).Google Scholar
Cross Ref
- Sandeep Shah and Jon G. Elerath. 2005. Reliability analysis of disk drive failure mechanisms. In Proceedings of the Annual Reliability and Maintainability Symposium. IEEE, 226--231.Google Scholar
- Y. Shiroishi, K. Fukuda, I. Tagawa, H. Iwasaki, S. Takenoiri, H. Tanaka, H. Mutoh, and N. Yoshikawa. 2009. Future options for HDD storage. IEEE Trans. Magnetics 45, 10 (Oct. 2009), 3816--3822.Google Scholar
Cross Ref
- Shuhei Tanakamaru, Yuki Yanagihara, and Ken Takeuchi. 2013. Error-prediction LDPC and error-recovery schemes for highly reliable solid-state drives (SSDs). IEEE J. Solid-State Circ. 48, 11 (2013), 2920--2933.Google Scholar
Cross Ref
- D. Weller, G. Parker, O. Mosendz, E. Champion, B. Stipe, X. Wang, T. Klemmer, G. Ju, and A. Aian. 2014. A HAMR media technology roadmap to an areal density of 4 Tb/in<sup>2</sup>. IEEE Trans. Magnetics 50, 1 (Jan. 2014).Google Scholar
Cross Ref
- S. B. Wicker and V. K. Bhargava. 1994. Reed-Solomon Codes and Their Applications. IEEE Press. Google Scholar
Digital Library
- R. Wood, R. Galbraith, and J. Coker. 2015. 2-D magnetic recording: Progress and evolution. IEEE Trans. Magnetics 51, 4 (April 2015), 1--7.Google Scholar
Cross Ref
- K. Zhao, W. Zhao, H. Sun, T. Zhang, X. Zhang, and N. Zheng. 2013. LDPC-in-SSD: Making advanced error correction codes work effectively in solid state drives. In Proceedings of the USENIX Conference on File and Storage Technologies. 243--256. Google Scholar
Digital Library
Index Terms
An Exploratory Study on Software-Defined Data Center Hard Disk Drives
Recommendations
Hard Disk Drives: The Good, the Bad and the Ugly!: HDDs are like the bread in a peanut butter and jelly sandwich.
File Systems and StorageHDDs are like the bread in a peanut butter and jelly sandwich—sort of an unexciting piece of hardware necessary to hold the “software.” They are simply a means to an end. HDD reliability, however, has always been a significant weak link, perhaps the ...
Study on flow-induced vibration of head–disk assembly mechanisms in actual hard disk drive
ASME-ISPS/JSME-IIP joint conference on Micromechatronics for Information and Precision Equipment, Santa Clara, California, USA, 2006Achieving ultra-high magnetic-recording density in hard disk drives (HDDs) requires clarification of flow-induced vibration issues. In particular, it is necessary to reduce the flow-induced disk vibration called disk flutter. Thus far, however, there ...
Study on flow-induced vibration of head---disk assembly mechanisms in actual hard disk drive
Achieving ultra-high magnetic-recording density in hard disk drives (HDDs) requires clarification of flow-induced vibration issues. In particular, it is necessary to reduce the flow-induced disk vibration called disk flutter. Thus far, however, there ...






Comments