ABSTRACTExponential growth in storage requirements and an increasing number of heterogeneous devices and application policies are making enterprise storage management a nightmare for administrators. Back-of-the-envelope calculations, rules of thumb, and manual correlation of individual device data are too error prone for the day-to-day administrative tasks of resource provisioning, problem determination, performance management, and impact analysis. Storage management tools have evolved over the past several years from standardizing the data reported by storage subsystems to providing intelligent planners. In this paper, we describe that evolution in the context of the IBM Total Storage® Productivity Center (TPC)--a suite of tools to assist administrators in the day-to-day tasks of monitoring, configuring, provisioning, managing change, analyzing configuration, managing performance, and determining problems. We describe our ongoing research to develop ways to simplify and automate these tasks by applying advanced analytics on the performance statistics and raw configuration and event data collected by TPC using the popular Storage Management Initiative-Specification (SMI-S). In addition, we provide details of SMART (storage management analytics and reasoning technology) as a library that provides a collection of data-aggregation functions and optimization algorithms.
AUTHORS
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of S. Gopisetty | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of S. Agarwala | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of E. Butler | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of D. Jadav | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of S. Jaquet | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of M. Korupolu | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of R. Routray | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of P. Sarkar | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of A. Singh | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of M. Sivan-Zimet | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of C.-H. Tan | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of S. Uttamchandani | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of D. Merbach | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of S. Padbidri | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of A. Dieberger | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of E. M. Haber | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of E. Kandogan | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of C. A. Kieliszewski | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of D. Agrawal | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of M. Devarakonda | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of K.-W. Lee | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of K. Magoutis | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of D. C. Verma | |||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of N. G. Vogl | |||||||||||||||||||||||||||||||||||||||||
REFERENCESNote: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.
|
1
|
Distributed Management Task Force, Inc., Common Information Model (CIM) Standards: see http:// www.dmtf.org/standards/cim/.
|
|
|
2
|
J. Case, M. Fedor, M. Schoffstall, and J. Davin, "A Simple Network Management Protocol (SNMP)," IETF Request for Comments 1098, Network Working Group (May 1990); see http://www.ietf.org/rfc/rfc/1157.txt.
|
|
|
3
|
Storage Networking Industry Association, SMI-S: The Storage Management Initiative Specification; see http:// www.snia.org/forums/smi/tech_programs/smis_home/.
|
|
|
4
|
The Open Group, SNIA Open Source Java CIM Object Managers (CIMOMs); see http://www.opengroup.org/ snia-cimom/.
|
|
|
5
|
IBM Corporation, IBM TotalStorage Productivity Center Suite; see http://www-304.ibm.com/jct03004c/systems/storage/ software/center/.
|
|
|
6
|
EMC Corporation, EMC ControlCenter Family; see http:// www.emc.com/products/family/controlcenter-family.htm.
|
|
|
7
|
Hewlett-Packard Development Company, HP Systems Insight Manager; see http://h18000.www1.hp.com/products/servers/ management/hpsim/index.html.
|
|
|
8
|
||
|
9
|
||
|
10
|
||
|
11
|
||
|
12
|
Li Yin , Sandeep Uttamchandani , John Palmer , Randy Katz , Gul Agha, AUTOLOOP: Automated Action Selection in the "Observe-Analyze-Act" Loop for Storage Systems, Proceedings of the Sixth IEEE International Workshop on Policies for Distributed Systems and Networks, p.129-138, June 06-08, 2005 [doi>10.1109/POLICY.2005.9]
|
|
|
13
|
||
|
14
|
J. Gray, "Why Do Computers Stop and What Can Be Done About it?" Proceedings of the Fifth Symposium on Reliability in Distributed Software and Database Systems, Los Angeles, CA, 1986, pp. 3-12.
|
|
|
15
|
||
|
16
|
M. Lovelace, M. Defiebre, H. Gunatilaka, C. Neal, and Y. Xu, "TotalStorage Productivity Center V3.3 Update Guide," IBM Redbooks, April 23, 2008; see http://www.redhooks.ibm. com/abstracts/sg247490.html.
|
|
|
17
|
||
|
18
|
S. Gopisetty , E. Butler , S. Jaquet , M. Korupolu , T. K. Nayak , R. Routray , M. Seaman , A. Singh , C.-H. Tan , S. Uttamchandani , A. Verma, Automated planners for storage provisioning and disaster recovery, IBM Journal of Research and Development, v.52 n.4, p.353-365, July 2008 [doi>10.1147/rd.524.0353]
|
|
|
19
|
A. Singh, M. Korupolu, and B. Bamba, "SPARK: Integrated Resource Allocation in Virtualization-Enabled SAN Data Centers," Research Report RJ-10407, IBM Almaden Research Center, San Jose, CA 95120, April 2007; see http://www. aameeksingh.com/projects/spark-rj10407.pdf
|
|
|
20
|
J. H. Friedman, "Multivariate Adaptive Regression Splines," Annals Statistics <b>19</b>, No. 1, 1-67 (1991).
|
|
|
21
|
L. Breiman, J. Friedman, C. J. Stone, and R. A. Olshen, Classification and Regression Trees, CRC Press, Boca Raton, FL, 2003.
|
|
|
22
|
J. R. Quinlan, "Learning with Continuous Classes," Proceedings of the Fifth Australian Joint Conference on Artificial Intelligence, Hobart, Tasmania, 1992, pp. 343-348.
|
|
|
23
|
P. J. Brockwell and R. A. Davis, Introduction to Time Series and Forecasting, Second Edition, Springer Publishing, New York, 2002.
|
|
|
24
|
J. W. Cooley and J. W. Tukey, "An Algorithm for the Machine Calculation of Complex Fourier Series," Math. Comput. <b>19</b>, No. 90, 297-301 (1965).
|
|
| |
25
|
|
|
26
|
Sandip Agarwala , Fernando Alegre , Karsten Schwan , Jegannathan Mehalingham, E2EProf: Automated End-to-End Performance Management for Enterprise Systems, Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, p.749-758, June 25-28, 2007 [doi>10.1109/DSN.2007.38]
|
|
|
27
|
||
|
28
|
||
|
29
|
A. Brown, G. Kar, and A. Keller, "An Active Approach to Characterizing Dynamic Dependencies for Problem Determination in a Distributed Environment," Proceedings of the Seventh IFIP/IEEE International Symposium on Integrated Network Management, Seattle, WA, 2001, pp. 377-390.
|
|
| |
30
|
Paramvir Bahl , Ranveer Chandra , Albert Greenberg , Srikanth Kandula , David A. Maltz , Ming Zhang, Towards highly reliable enterprise network services via inference of multi-level dependencies, Proceedings of the 2007 conference on Applications, technologies, architectures, and protocols for computer communications, August 27-31, 2007, Kyoto, Japan [doi>10.1145/1282380.1282383]
|
|
31
|
J. B. MacQueen, "Some Methods for Classification and Analysis of MultiVariate Observations," Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, 1967, pp. 281-297.
|
|
|
32
|
A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum Likelihood from Incomplete Data Via the EM Algorithm," J. R. Stat. Soc. Series B (Methodological) <b>39</b>, No. 1, 1977, 1-38 (1977).
|
|
|
33
|
CA, Inc., CA Storage Resource Manager; see http://ca.com/us/ products/product.aspx?ID=1541.
|
|
|
34
|
Hewlett-Packard Development Company, HP Storage Essentials (SRM) Software; see http://h18006.www1.hp.com/ storage/software/srmgt/index.html.
|
|
|
35
|
Symantec Corporation, Storage Management; see http:// www.symantec.com/business/products/category.jsp?pcid=2245.
|
|
|
36
|
NetApp, Inc., NetApp Management Software: Storage Suite; see http://www.netapp.com/us/products/management-software/.
|
|
|
37
|
D. Russell and R. E. Passmore, "Magic Quadrant for Storage Resource Management and SAN Management Software, 2007," Technical Report G00146578, Gartner RAS Core Research, Gartner, March 2007.
|
|
| |
38
|
|
|
39
|
ITIL: IT Infrastructure Library; see http://www.itil-officialsite. com/home/home.asp.
|
|
|
40
|
||
|
41
|
CA, Inc., CA SAN Designer; see http://ca.com/us/products/ product.aspx?ID=4590.
|
|
| |
42
|
Guillermo A. Alvarez , Elizabeth Borowsky , Susie Go , Theodore H. Romer , Ralph Becker-Szendy , Richard Golding , Arif Merchant , Mirjana Spasojevic , Alistair Veitch , John Wilkes, Minerva: An automated resource provisioning tool for large-scale storage systems, ACM Transactions on Computer Systems (TOCS), v.19 n.4, p.483-518, November 2001 [doi>10.1145/502912.502915]
|
| |
43
|
|
|
44
|
||
| |
45
|
|
|
46
|
S. Agarwala, R. Routray, and S. Uttamchandani, "ChargeView: An Integrated Tool for Implementing Chargeback in IT Systems," Proceedings of the 11th IEEE/ IFIP Network Operations and Management Symposium, Salvador, Bahia, Brazil, 2008, see http://www.iit.edu/ ~routram/ChargeView.pdf.
|
|
|
47
|
S. Uttamchandani, K. Voruganti, R. Routray, L. Yin, A. Singh, and B. Yolken, "BRAHMA: Planning Tool for Providing Storage Management as a Service," Proceedings of the IEEE International Conference on Services Computing, Salt Lake City, UT, 2007, pp. 1-10.
|
|
|
48
|
CITED BY5 Citations
|
|
|
|
|
|
INDEX TERMSThe ACM Computing Classification System (CCS rev.2012)
PUBLICATION| Title | IBM Journal of Research and Development table of contents archive |
| Volume 52 Issue 4, July 2008 | |
| Pages | 341-352 |
| Publication Date | 2008-07-01 (yyyy-mm-dd) |
| Publisher | IBM Corp. Riverton, NJ, USA |
| ISSN: 0018-8646 doi>10.1147/rd.524.0341 |
REVIEWS
COMMENTSBe the first to comment To Post a comment please sign in or create a free Web account
Table of ContentsVolume 52 Issue 4, July 2008
| Preface | |
| Dilip D. Kandlur, Chandrasekhar Narayan | |
| Pages: 317-318 | |
| doi>10.1147/rd.524.0317 | |
| IBM scale out file services: reinventing network-attached storage | |
| S. Oehme, J. Deicke, J.-P. Akelbein, R. Sahlberg, A. Tridgell, R. L. Haskin | |
| Pages: 319-328 | |
| doi>10.1147/rd.524.0319 | |
|
This paper focuses on the challenges of creating a scalable network-attached storage (NAS) filer with hundreds of nodes on top of the IBM General Parallel File System™ (GPFS™). The filer is created as a multiheaded data exporter over Ethernet ...
expand
|
|
| Glamor: an architecture for file system federation | |
| U. Lanjewar, M. Naik, R. Tewari | |
| Pages: 329-339 | |
| doi>10.1147/rd.524.0329 | |
|
We present the design and implementation of Glamor, a framework for building a wide-area federated file system. Glamor enables distributed file access across enterprise-wide or Internet-scale networks where users can seamlessly navigate through data ...
expand
|
|
| Evolution of storage management: transforming raw data into information | |
| S. Gopisetty, S. Agarwala, E. Butler, D. Jadav, S. Jaquet, M. Korupolu, R. Routray, P. Sarkar, A. Singh, M. Sivan-Zimet, C.-H. Tan, S. Uttamchandani, D. Merbach, S. Padbidri, A. Dieberger, E. M. Haber, E. Kandogan, C. A. Kieliszewski, D. Agrawal, M. Devarakonda, K.-W. Lee, K. Magoutis, D. C. Verma, N. G. Vogl | |
| Pages: 341-352 | |
| doi>10.1147/rd.524.0341 | |
|
Exponential growth in storage requirements and an increasing number of heterogeneous devices and application policies are making enterprise storage management a nightmare for administrators. Back-of-the-envelope calculations, rules of thumb, and manual ...
expand
|
|
| Automated planners for storage provisioning and disaster recovery | |
| S. Gopisetty, E. Butler, S. Jaquet, M. Korupolu, T. K. Nayak, R. Routray, M. Seaman, A. Singh, C.-H. Tan, S. Uttamchandani, A. Verma | |
| Pages: 353-365 | |
| doi>10.1147/rd.524.0353 | |
|
Introducing an application into a data center involves complex interrelated decision-making for the placement of data (where to store it) and resiliency in the event of a disaster (how to protect it). Automated planners can assist administrators in making ...
expand
|
|
| Galapagos: model-driven discovery of end-to-end application-storage relationships in distributed systems | |
| K. Magoutis, M. Devarakonda, N. Joukov, N. G. Vogl | |
| Pages: 367-377 | |
| doi>10.1147/rd.524.0367 | |
|
Modern business information systems are typically multitiered distributed systems comprising Web services, application services, databases, enterprise information systems, file systems, storage controllers, and other storage systems. In such environments, ...
expand
|
|
| Archive storage system design for long-term storage of massive amounts of data | |
| P. L. Bradshaw, K. W. Brannon, T. Clark, K. Dahman, S. Doraiswamy, L. Duyanovich, B. L. Hillsberg, W. Hineman, M. Kaczmarski, B. J. Klingenberg, X. Ma, R. Rees | |
| Pages: 379-388 | |
| doi>10.1147/rd.524.0379 | |
|
A dramatic shift is underway in how organizations use computer storage. This shift will have a profound impact on storage system design. The requirement for storage of traditional transactional data is being supplemented by the necessity to store information ...
expand
|
|
| Preservation DataStores: new storage paradigm for preservation environments | |
| S. Rabinovici-Cohen, M. E. Factor, D. Naor, L. Ramati, P. Reshef, S. Ronen, J. Satran, D. L. Giaretta | |
| Pages: 389-399 | |
| doi>10.1147/rd.524.0389 | |
|
As the world becomes digital, we are in ever greater danger of losing business, scientific, artistic, cultural, and personal assets. The threat of such a digital dark age stems from the fact that-- unlike physical records that may survive decades, centuries, ...
expand
|
|
| The ANSI T10 object-based storage standard and current implementations | |
| D. Nagle, M. E. Factor, S. Iren, D. Naor, E. Riedel, O. Rodeh, J. Satran | |
| Pages: 401-411 | |
| doi>10.1147/rd.524.0401 | |
|
Object-based storage is the natural evolution of the block storage interface, aimed at efficiently and effectively meeting the performance, reliability, security, and service requirements demanded by current and future applications. The object-based ...
expand
|
|
| Undetected disk errors in RAID arrays | |
| J. L. Hafner, V. Deenadhayalan, W. Belluomini, K. Rao | |
| Pages: 413-425 | |
| doi>10.1147/rd.524.0413 | |
|
Though remarkably reliable, disk drives do fail occasionally. Most failures can be detected immediately; moreover, such, failures can be modeled and addressed using technologies such as RAID (Redundant Arrays of Independent Disks). Unfortunately, disk ...
expand
|
|
| An architecture for storage-hosted application extensions | |
| D. Chambliss, P. Pandey, T. Thakur, A. Fleshler, T. Clark, J. A. Ruddy, K. D. Gougherty, M. Kalos, L. Merithew, J. G. Thompson, H. M. Yudenfriend | |
| Pages: 427-437 | |
| doi>10.1147/rd.524.0427 | |
|
The very rapid growth of data-intensive computing makes it attractive to perform computations locally, where data is stored. Large storage systems based on standard system technologies with server virtualization capabilities make it feasible to deploy ...
expand
|
|
| Storage-class memory: the next storage system technology | |
| R. F. Freitas, W. W. Wilcke | |
| Pages: 439-447 | |
| doi>10.1147/rd.524.0439 | |
|
The dream of replacing rotating mechanical storage, the disk drive, with solid-state, nonvolatile RAM may become a reality in the near future. Approximately ten new technologies--collectively called storage-class memory (SCM)--are currently under development ...
expand
|
|
| Overview of candidate device technologies for storage-class memory | |
| G. W. Burr, B. N. Kurdi, J. C. Scott, C. H. Lam, K. Gopalakrishnan, R. S. Shenoy | |
| Pages: 449-464 | |
| doi>10.1147/rd.524.0449 | |
|
Storage-class memory (SCM) combines the benefits of a solid-state memory, such as high performance and robustness, with the archival capabilities and low cost of conventional hard-disk magnetic storage. Such a device would require a solid-state nonvolatile ...
expand
|
|
| Phase-change random access memory: a scalable technology | |
| S. Raoux, G. W. Burr, M. J. Breitwisch, C. T. Rettner, Y.-C. Chen, R. M. Shelby, M. Salinga, D. Krebs, S.-H. Chen, H.-L. Lung, C. H. Lam | |
| Pages: 465-479 | |
| doi>10.1147/rd.524.0465 | |
|
Nonvolatile RAM using resistance contrast in phase-change materials [or phase-change RAM (PCRAM)] is a promising technology for future storage-class memory. However, such a technology can succeed only if it can scale smaller in size, given the increasingly ...
expand
|
|
| Transition-metal-oxide-based resistance-change memories | |
| S. F. Karg, G. I. Meijer, J. G. Bednorz, C. T. Rettner, A. G. Schrott, E. A. Joseph, C. H. Lam, M. Janousch, U. Staub, F. La Mattina, S. F. Alvarado, D. Widmer, R. Stutz, U. Drechsler, D. Caimi | |
| Pages: 481-492 | |
| doi>10.1147/rd.524.0481 | |
|
We provide a status report on the development of perovskite-based transition-metal-oxide resistance-change memories. We focus on bipolar resistance switching observed in Cr-doped SrTiO3 memory cells with dimensions ranging from bulk single ...
expand
|
|
| Probe-based ultrahigh-density storage technology | |
| A. Pantazi, A. Sebastian, T. A. Antonakopoulos, P. Bächtold, A. R. Bonaccio, J. Bonan, G. Cherubini, M. Despont, R. A. DiPietro, U. Drechsler, U. Dürig, B. Gotsmann, W. Häberle, C. Hagleitner, J. L. Hedrick, D. Jubin, A. Knoll, M. A. Lantz, J. Pentarakis, H. Pozidis, R. C. Pratt, H. Rothuizen, R. Stutz, M. varsamou, D. Wiesmann, E. Eleftheriou | |
| Pages: 493-511 | |
| doi>10.1147/rd.524.0493 | |
|
Ultrahigh storage densities can be achieved by using a thermomechanical scanning-probe-based data-storage approach to write, read back, and erase data in very thin polymer films. High data rates are achieved by parallel operation of large two-dimensional ...
expand
|
|
| Scaling tape-recording areal densities to 100 Gb/in 2 | |
| A. J. Argumedo, D. Berman, R. G. Biskeborn, G. Cherubini, R. D. Cideciyan, E. Eleftheriou, W. Häberle, D. J. Hellman, R. Hutchins, W. Imaino, J. Jelitto, K. Judd, P.-O. Jubert, M. A. Lantz, G. M. McClelland, T. Mittelholzer, C. Narayan, S. Ölçer, P. J. Seger | |
| Pages: 513-527 | |
| doi>10.1147/rd.524.0513 | |
|
We examine the issue of scaling magnetic tape-recording to higher areal densities, focusing on the challenges of achieving 100 Gb/in2 in the linear tape format. The current highest achieved areal density demonstrations of 6.7 Gb/in2 ...
expand
|
|
| Flash memories: successes and challenges | |
| S. K. Lai | |
| Pages: 529-535 | |
| doi>10.1147/rd.524.0529 | |
|
Flash memory grew from a simple concept in the early 1980s to a technology that generated close to $23 billion in worldwide revenue in 2007, and this represents one of the many success stories in the semiconductor industry. This success was made possible ...
expand
|