skip to main content
research-article

EMPRESS: Accelerating Scientific Discovery through Descriptive Metadata Management

Authors Info & Claims
Published:12 December 2022Publication History
Skip Abstract Section

Abstract

High-performance computing scientists are producing unprecedented volumes of data that take a long time to load for analysis. However, many analyses only require loading in the data containing particular features of interest and scientists have many approaches for identifying these features. Therefore, if scientists store information (descriptive metadata) about these identified features, then for subsequent analyses they can use this information to only read in the data containing these features. This can greatly reduce the amount of data that scientists have to read in, thereby accelerating analysis. Despite the potential benefits of descriptive metadata management, no prior work has created a descriptive metadata system that can help scientists working with a wide range of applications and analyses to restrict their reads to data containing features of interest. In this article, we present EMPRESS, the first such solution. EMPRESS offers all of the features needed to help accelerate discovery: It can accelerate analysis by up to 300 ×, supports a wide range of applications and analyses, is high-performing, is highly scalable, and requires minimal storage space. In addition, EMPRESS offers features required for a production-oriented system: scalable metadata consistency techniques, flexible system configurations, fault tolerance as a service, and portability.

REFERENCES

  1. [1] Abbasi Hasan, Wolf Matthew, Eisenhauer Greg, Klasky Scott, Schwan Karsten, and Zheng Fang. 2010. DataStager: Scalable data staging services for petascale applications. Clust. Comput. 13, 3 (2010), 277290.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Albrand Solveig, Doherty Thomas, Fulachier Jerome, and Lambert Fabian. 2008. The ATLAS metadata interface. In Journal of Physics: Conference Series, Vol. 119. IOP Publishing, Bristol, United Kingdom.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Alger Matthew J., Banfield Julie K., Ong Cheng Soon, Rudnick Lawrence, Wong O. I., Wolf Christian, Andernach Heinz, Norris Ray P., and Shabala Stanislav S.. 2018. Radio Galaxy Zoo: Machine learning for radio source host galaxy cross-identification. Month. Not. Roy. Astron. Societ. 478, 4 (2018), 55475563.Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Aliabadi Shahrouz, Johnson Andrew, Abedi Jalal, and Zellars Bruce. 2002. High performance computing of fluid-structure interactions in hydrodynamics applications using unstructured meshes with more than one billion elements. In International Conference on High-performance Computing. Springer, New York, NY, 519533.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Auricchio Ferdinando, Fedele Marco, Ferretti Marco, Lefieux Adrien, Romarowski Rodrigo, Santangelo Luigi, and Veneziani Alessandro. 2018. Benchmarking a hemodynamics application on Intel based HPC systems. Parallel Comput. Everyw. 32 (2018), 57.Google ScholarGoogle Scholar
  6. [6] Balaji Pavan, Buntinas Darius, Goodell David, Gropp William, Hoefler Torsten, Kumar Sameer, Lusk Ewing, Thakur Rajeev, and Träff Jesper Larsson. 2011. MPI on millions of cores. Parallel Process. Lett. 21, 1 (2011), 4560.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Banesh Divya, Wendelberger Joanne, Petersen Mark, Ahrens James, and Hamann Bernd. 2018. Change point detection for ocean eddy analysis. In Workshop on Visualisation in Environmental Sciences (EnvirVis). The Eurographics Association, 2733. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Barone Matthew F., Ray Jaideep, and Domino Stefan. 2021. Feature selection, clustering, and prototype placement for turbulence data sets. In AIAA SciTech 2021 Forum. American Institute of Aeronautics and Astronautic. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Bennett Janine C., Abbasi Hasan, Bremer Peer-Timo, Grout Ray, Gyulassy Attila, Jin Tong, Klasky Scott, Kolla Hemanth, Parashar Manish, Pascucci Valerio, et al. 2012. Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In International Conference on High Performance Computing, Networking, Storage and Analysis. IEEE, New York, NY, 19.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Berrocal Eduardo, Bautista-Gomez Leonardo, Di Sheng, Lan Zhiling, and Cappello Franck. 2016. Exploring partial replication to improve lightweight silent data corruption detection for HPC applications. In European Conference on Parallel Processing. Springer, New York, NY, 419430.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Betoule M., Marriner J., Regnault N., Cuillandre J.-C., Astier P., Guy J., Balland C., Hage P. El, Hardin D., Kessler R., et al. 2013. Improved photometric calibration of the SNLS and the SDSS supernova surveys. Astron. Astrophys. 552 (2013), A124.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Beyer Johanna, Al-Awami Ali, Kasthuri Narayanan, Lichtman Jeff W., Pfister Hanspeter, and Hadwiger Markus. 2013. ConnectomeExplorer: Query-guided visual analysis of large volumetric neuroscience data. IEEE Trans. Visualiz. Comput. Graph. 19, 12 (2013), 28682877.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Biswas Ayan, Dutta Soumya, Pulido Jesus, and Ahrens James. 2018. In situ data-driven adaptive sampling for large-scale simulation data summarization. In Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization. ACM, New York, NY, 1318.Google ScholarGoogle Scholar
  14. [14] Biswas Manish and Adlak Ritesh. 2018. Classification of galaxy morphologies using artificial neural network. In 4th International Conference for Convergence in Technology (I2CT). IEEE, New York, NY, 14.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Bonnie David John. 2018. GUFI Overview. Technical Report. Los Alamos National Lab (LANL), Los Alamos, NM.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Bremer Peer-Timo, Weber Gunther, Tierny Julien, Pascucci Valerio, Day Marc, and Bell John. 2010. Interactive exploration and analysis of large-scale simulations using topology-based data segmentation. IEEE Trans. Visualiz. Comput. Graph. 17, 9 (2010), 13071324.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Byna Suren, Chaarawi Mohamad, Koziol Quincey, Mainzer John, and Willmore Frank. 2017. Tuning HDF5 subfiling performance on parallel file systems. In Cray User Group Conference (CUG’17).Google ScholarGoogle Scholar
  18. [18] Carr Hamish and Snoeyink Jack. 2003. Path seeds and flexible isosurfaces using topology for exploratory visualization. In Symposium on Data Visualisation. The Eurographics Association, 4958.Google ScholarGoogle Scholar
  19. [19] Chen Hsing-bung (H. B.), Grider Gary, and Montoya David. 2017. An early functional and performance experiment of the MarFS Hybrid Storage EcoSystem. In IEEE International Conference on Cloud Engineering (IC2E). 5966. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Chiu Hsuan-Te, Chou Jerry, Vishwanath Venkat, and Wu Kesheng. 2015. In-memory query system for scientific datasets. In IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS). IEEE, New York, NY, 362371.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Franzone Piero Colli, Pavarino Luca F., and Scacchi Simone. 2018. A numerical study of scalable cardiac electro-mechanical solvers on HPC architectures. Front. Physiol. 9 (2018), 268.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Cubuk Ekin D., Schoenholz Samuel Stern, Rieser Jennifer M., Malone Brad Dean, Rottler Joerg, Durian Douglas J., Kaxiras Efthimios, and Liu Andrea J.. 2015. Identifying structural flow defects in disordered solids using machine-learning methods. Phys. Rev. Lett. 114, 10 (2015), 108001.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Dayal Jai and Lofstead Jay. 2016. Doubly Distributed Transactions. Retrieved from https://github.com/gflofst/d2t.Google ScholarGoogle Scholar
  24. [24] Deelman Ewa, Singh Gurmeet, Su Mei-Hui, Blythe James, Gil Yolanda, Kesselman Carl, Mehta Gaurang, Vahi Karan, Berriman G. Bruce, Good John, et al. 2005. Pegasus: A framework for mapping complex scientific workflows onto distributed systems. Sci. Program. 13, 3 (2005), 219237.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Deutsch Peter and Gailly J.. 1996. ZLIB Compressed Data Format Specification Version 3.3. Technical Report. RFC 1950, May.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Devine Thomas Ryan, Goseva-Popstojanova Katerina, and McLaughlin Maura. 2016. Detection of dispersed radio pulses: A machine learning approach to candidate identification and classification. Month. Not. Roy. Astron. Societ. 459, 2 (2016), 15191532.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Dewdney Peter, Hall Peter, Schillizzi R., and Lazio J.. 2009. The Square Kilometre Array. Proc. Instit. Electric. Electron. Eng. 97, 8 (2009), 14821496.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Di Sheng and Cappello Franck. 2017. Optimization of error-bounded lossy compression for hard-to-compress HPC data. IEEE Trans. Parallel Distrib. Syst. 29, 1 (2017), 129143.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Docan Ciprian, Parashar Manish, and Klasky Scott. 2012. DataSpaces: An interaction and coordination framework for coupled simulation workflows. Clust. Comput. 15, 2 (2012), 163181.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Dorier Matthieu, Sisneros Robert, Gomez Leonardo Bautista, Peterka Tom, Orf Leigh, Rahmani Lokman, Antoniu Gabriel, and Bougé Luc. 2016. Adaptive performance-constrained in situ visualization of atmospheric simulations. In IEEE International Conference on Cluster Computing (CLUSTER). IEEE, New York, NY, 269278.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Feki Saber and Gabriel Edgar. 2008. Incorporating historic knowledge into a communication library for self-optimizing high performance computing applications. In 2nd IEEE International Conference on Self-adaptive and Self-organizing Systems. IEEE, New York, NY, 265274.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Folk Mike, Heber Gerd, Koziol Quincey, Pourmal Elena, and Robinson Dana. 2011. An overview of the HDF5 technology suite and its applications. In EDBT/ICDT Workshop on Array Databases. ACM, New York, NY, 3647.Google ScholarGoogle Scholar
  33. [33] Forum MPI. 2012. MPI: A Message-Passing Interface Standard Version 3.0. Retrieved November 20, 2022 from https://www.mpi-forum.org/mpi-30/.Google ScholarGoogle Scholar
  34. [34] Fujishiro Issei, Otsuka Rieko, Takahashi Shigeo, and Takeshima Yuriko. 2005. T-Map: A topological approach to visual exploration of time-varying volume data. In High-Performance Computing. Springer, New York, NY, 176190.Google ScholarGoogle Scholar
  35. [35] Gabriel Edgar, Feki Saber, Benkert Katharina, and Resch Michael M.. 2010. Towards performance portability through runtime adaptation for high-performance computing applications. Concurr. Computat.: Pract. Exper. 22, 16 (2010), 22302246.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Gahvari Hormozd, Baker Allison H., Schulz Martin, Yang Ulrike Meier, Jordan Kirk E., and Gropp William. 2011. Modeling the performance of an algebraic multigrid cycle on HPC platforms. In International Conference on Supercomputing. ACM, New York, NY, 172181.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Garcia-Gasulla Marta, Mantovani Filippo, Josep-Fabrego Marc, Eguzkitza Beatriz, and Houzeaux Guillaume. 2020. Runtime mechanisms to survive new HPC architectures: A use case in human respiratory simulations. Int. J. High Perform. Comput. Applic. 34, 1 (2020), 4256.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Gauci Adam, Adami Kristian Zarb, and Abela John. 2010. Machine learning for galaxy morphology classification. arXiv preprint arXiv:1005.0390 (2010).Google ScholarGoogle Scholar
  39. [39] Gong Zhenhuan, II David A. Boyuka, Zou Xiaocheng, Liu Qing, Podhorszki Norbert, Klasky Scott, Ma Xiaosong, and Samatova Nagiza F.. 2013. PARLO: PArallel Run-time Layout Optimization for scientific data explorations with heterogeneous access patterns. In 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid). IEEE, New York, NY, 343351.Google ScholarGoogle Scholar
  40. [40] Gong Zhenhuan, Lakshminarasimhan Sriram, Jenkins John, Kolla Hemanth, Ethier Stephane, Chen Jackie, Ross Robert, Klasky Scott, and Samatova Nagiza F.. 2012. Multi-level layout optimization for efficient spatio-temporal queries on ISABELA-compressed data. In IEEE 26th International Symposium on Parallel & Distributed Processing Symposium (IPDPS). IEEE, New York, NY, 873884.Google ScholarGoogle Scholar
  41. [41] Gong Zhenhuan, Rogers Terry, Jenkins John, Kolla Hemanth, Ethier Stephane, Chen Jackie, Ross Robert, Klasky Scott, and Samatova Nagiza F.. 2012. MLOC: Multi-level layout optimization framework for compressed scientific data exploration with heterogeneous access patterns. In 41st International Conference on Parallel Processing. IEEE, New York, NY, 239248.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Gosink Luke, Anderson John, Bethel Wes, and Joy Kenneth. 2007. Variable interactions in query-driven visualization. IEEE Trans. Visualiz. Comput. Graph. 13, 6 (2007), 14001407.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Gosink Luke, Shalf John, Stockinger Kurt, Wu Kesheng, and Bethel Wes. 2006. HDF5-FastQuery: Accelerating complex queries on HDF datasets using fast bitmap indices. In 18th International Conference on Scientific and Statistical Database Management. IEEE, New York, NY, 149158.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Gosink Luke J., Garth Christoph, Anderson John C., Bethel E. Wes, and Joy Kenneth I.. 2010. An application of multivariate statistical analysis for query-driven visualization. IEEE Trans. Visualiz. Comput. Graph. 17, 3 (2010), 264275.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Graff Philip, Feroz Farhan, Hobson Michael P., and Lasenby Anthony. 2014. SKYNET: An efficient and robust neural network training tool for machine learning in astronomy. Month. Not. Roy. Astron. Societ. 441, 2 (2014), 17411759.Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Grawinkel Matthias, Nagel Lars, Mäsker Markus, Padua Federico, Brinkmann André, and Sorth Lennart. 2015. Analysis of the ECMWF storage landscape. In 13th USENIX Conference on File and Storage Technologies (FAST’15). USENIX Association, Berkeley, CA, 1527.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. [47] Gropp William, Gropp William D., Lusk Ewing, Skjellum Anthony, and Lusk Argonne Distinguished Fellow Emeritus Ewing. 1999. Using MPI: Portable Parallel Programming with the Message-passing Interface. Vol. 1. MIT Press, Cambridge, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Gropp William, Hoefler Torsten, Thakur Rajeev, and Lusk Ewing. 2014. Using Advanced MPI: Modern Features of the Message-passing Interface. MIT Press.Google ScholarGoogle Scholar
  49. [49] Gropp William, Thakur Rajeev, and Lusk Ewing. 1999. Using MPI-2: Advanced Features of the Message-passing Interface. MIT Press, Cambridge, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Gu Junmin, Klasky Scott, Podhorszki Norbert, Qiang Ji, and Wu Kesheng. 2018. Querying large scientific data sets with adaptable IO system ADIOS. In Asian Conference on Supercomputing Frontiers. Springer, New York, NY, 5169.Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Gyulassy Attila and Natarajan Vijay. 2005. Topology-based simplification for feature extraction from 3D scalar fields. In IEEE Visualization Conference. IEEE, New York, NY.Google ScholarGoogle Scholar
  52. [52] Habib Salman, Roser Robert, Gerber Richard, Antypas Katie, Riley Katherine, Williams Tim, Wells Jack, Straatsma Tjerk, Almgren A., Amundson J., et al. 2016. ASCR/HEP exascale requirements review report. arXiv preprint arXiv:1603.09303 (2016).Google ScholarGoogle Scholar
  53. [53] Hong Seungkyun, Kim Seongchan, Joh Minsu, and Song Sa-kwang. 2017. GlobeNet: Convolutional neural networks for typhoon eye tracking from remote sensing imagery. arXiv preprint arXiv:1708.03417 (2017).Google ScholarGoogle Scholar
  54. [54] Hsu Ming-Chen, Akkerman Ido, and Bazilevs Yuri. 2011. High-performance computing of wind turbine aerodynamics using isogeometric analysis. Comput. Fluids 49, 1 (2011), 93100.Google ScholarGoogle ScholarCross RefCross Ref
  55. [55] Ivezic Zeljko, Tyson J. A., Abel B., Acosta E., Allsman R., AlSayyad Y., Anderson S. F., Andrew J., Angel R., Angeli G., et al. 2008. LSST: From science drivers to reference design and anticipated data products. arXiv preprint arXiv:0805.2366 (2008).Google ScholarGoogle Scholar
  56. [56] JAMO 2018. JAMO—JGI Archive and Metadata Organizer. Retrieved from https://storageconference.us/2018/Presentations/Beecroft.pdf.Google ScholarGoogle Scholar
  57. [57] Jenkins John, Arkatkar Isha, Lakshminarasimhan Sriram, Shah Neil, Schendel Eric R., Ethier Stephane, Chang Choong-Seock, Chen Jacqueline H., Kolla Hemanth, Klasky Scott, et al. 2012. Analytics-driven lossless data compression for rapid in-situ indexing, storing, and querying. In International Conference on Database and Expert Systems Applications. Springer, New York, NY, 1630.Google ScholarGoogle ScholarCross RefCross Ref
  58. [58] Jenkins J., Schendel E. R., Lakshminarasimhan S., Boyuka D. A., Rogers T., Ethier S., Ross R., Klasky S., and Samatova N. F.. 2012. Byte-precision level of detail processing for variable precision analytics. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC). 111. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. [59] Johnson Charles, Keeton Kimberly, III Charles B. Morrey, Soules Craig A. N., Veitch Alistair C., Bacon Stephen, Batuner Oskar, Condotta Marcelo, Coutinho Hamilton, Doyle Patrick J., et al. 2014. From research to practice: Experiences engineering a production metadata database for a scale out file system. In USENIX Conference on File and Storage Technologies. USENIX Association, Berkeley, CA, 191198.Google ScholarGoogle Scholar
  60. [60] Khan Awais, Kim Taeuk, Byun Hyunki, and Kim Youngjae. 2019. SciSpace: A scientific collaboration workspace for geo-distributed HPC data centers. Fut. Gen. Comput. Syst. 101 (2019), 398409.Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. [61] Korenblum Daniel, Rubin Daniel, Napel Sandy, Rodriguez Cesar, and Beaulieu Chris. 2011. Managing biomedical image metadata for search and retrieval of similar images. J. Digit. Imag. 24, 4 (2011), 739748.Google ScholarGoogle ScholarCross RefCross Ref
  62. [62] Ku S., Chang C. S., and Diamond P. H.. 2009. Full-f gyrokinetic particle simulation of centrally heated global ITG turbulence from magnetic axis to edge pedestal top in a realistic Tokamak geometry. Nucl. Fus. 49, 11 (2009), 115021.Google ScholarGoogle ScholarCross RefCross Ref
  63. [63] Kuhn Michael and Duwe Kira. 2020. Coupling storage systems and self-describing data formats for global metadata management. In International Conference on Computational Science and Computational Intelligence (CSCI). IEEE, New York, NY, 12241230.Google ScholarGoogle ScholarCross RefCross Ref
  64. [64] Lakshminarasimhan Sriram, Shah Neil, Ethier Stephane, Klasky Scott, Latham Rob, Ross Rob, and Samatova Nagiza F.. 2011. Compressing the incompressible with ISABELA: In-situ reduction of spatio-temporal data. In European Conference on Parallel Processing. Springer, New York, NY, 366379.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. [65] Lakshminarasimhan Sriram, Zou Xiaocheng, Boyuka David A., Pendse Saurabh V., Jenkins John, Vishwanath Venkatram, Papka Michael E., Klasky Scott, and Samatova Nagiza F.. 2014. DIRAQ: Scalable in situ data-and resource-aware indexing for optimized query performance. Clust. Comput. 17, 4 (2014), 11011119.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. [66] Landge Aaditya G., Pascucci Valerio, Gyulassy Attila, Bennett Janine C., Kolla Hemanth, Chen Jacqueline, and Bremer Peer-Timo. 2014. In-situ feature extraction of large scale combustion simulations using segmented merge trees. In International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, New York, NY, 10201031.Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. [67] Lawson Margaret. 2018. The Next Generation of EMPRESS: A Metadata Management System for Accelerated Scientific Discovery at Exascale. Bachelor’s Thesis. Dartmouth College.Google ScholarGoogle Scholar
  68. [68] Lawson Margaret and Lofstead Jay. 2018. Using a robust metadata management system to accelerate scientific discovery at extreme scales. In 3rd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISCS’18). IEEE, New York, NY, 1323. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  69. [69] Leung Andrew W., Shao Minglong, Bisson Timothy, Pasupathy Shankar, and Miller Ethan L.. 2009. Spyglass: Fast, scalable metadata search for large-scale storage systems. In USENIX Conference on File and Storage Technologies. 153166.Google ScholarGoogle Scholar
  70. [70] Li Jianwei, Liao Wei keng, Choudhary A., Ross R., Thakur R., Gropp W., Latham R., Siegel A., Gallagher B., and Zingale M.. 2003. Parallel netCDF: A high-performance scientific I/O interface. In ACM/IEEE Conference on Supercomputing. IEEE, New York, NY, 3939. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  71. [71] Liu Yunjie, Racah Evan, Correa Joaquin, Khosrowshahi Amir, Lavers David, Kunkel Kenneth, Wehner Michael, Collins William, et al. 2016. Application of deep convolutional neural networks for detecting extreme weather in climate datasets. arXiv preprint arXiv:1605.01156 (2016).Google ScholarGoogle Scholar
  72. [72] Livnat Yarden, Shen Han-Wei, and Johnson Christopher R.. 1996. A near optimal isosurface extraction algorithm using the span space. IEEE Trans. Visualiz. Comput. Graph. 2, 1 (1996), 7384.Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. [73] Lofstead Jay and Dayal Jai. 2012. Transactional parallel metadata services for integrated application workflows. In High Performance Computing Meets Databases at Supercomputing (HPCDB’12).Google ScholarGoogle Scholar
  74. [74] Lofstead Jay, Dayal Jai, Schwan Karsten, and Oldfield Ron. 2012. D2T: Doubly distributed transactions for high performance and distributed computing. In IEEE International Conference on Cluster Computing (CLUSTER). IEEE, New York, NY, 9098.Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. [75] Lofstead Jay, Jimenez Ivo, Maltzahn Carlos, Koziol Quincey, Bent John, and Barton Eric. 2016. DAOS and friends: A proposal for an exascale storage system. In International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, New York, NY, 585596.Google ScholarGoogle ScholarCross RefCross Ref
  76. [76] Lofstead Jay, Polte Milo, Gibson Garth, Klasky Scott, Schwan Karsten, Oldfield Ron, Wolf Matthew, and Liu Qing. 2011. Six degrees of scientific data: Reading patterns for extreme scale science IO. In 20th International Symposium on High-performance Distributed Computing (HPDC’11). ACM, New York, NY, 4960. DOI: http://doi.acm.org/10.1145/1996130.1996139Google ScholarGoogle Scholar
  77. [77] Lofstead Jay, Zheng Fang, Klasky Scott, and Schwan Karsten. 2009. Adaptable, metadata rich IO methods for portable high performance IO. In IEEE International Symposium on Parallel & Distributed Processing (IPDPS). IEEE, New York, NY, 110.Google ScholarGoogle Scholar
  78. [78] Lofstead Jay, Zheng Fang, Liu Qing, Klasky Scott, Oldfield Ron, Kordenbrock Todd, Schwan Karsten, and Wolf Matthew. 2010. Managing variability in the IO performance of petascale storage systems. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC). IEEE, New York, NY, 112.Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. [79] Lopez Jose Manuel, Feldmann Daniel, Rampp Markus, Vela-Martin Alberto, Shi Liang, and Avila Marc. 2019. nsCouette—A high-performance code for direct numerical simulations of turbulent Taylor-Couette flow. arXiv preprint arXiv:1908.00587 (2019).Google ScholarGoogle Scholar
  80. [80] Lorensen William E. and Cline Harvey E.. 1987. Marching cubes: A high resolution 3D surface construction algorithm. ACM SIGGRAPH 21, 4 (1987), 163169.Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. [81] Ludäscher Bertram, Altintas Ilkay, Berkley Chad, Higgins Dan, Jaeger Efrat, Jones Matthew, Lee Edward A., Tao Jing, and Zhao Yang. 2006. Scientific workflow management and the Kepler system. Concurr. Computat.: Pract. Exper. 18, 10 (2006), 10391065.Google ScholarGoogle ScholarCross RefCross Ref
  82. [82] Lüttgau Jakob, Kuhn Michael, Duwe Kira, Alforov Yevhen, Betke Eugen, Kunkel Julian, and Ludwig Thomas. 2018. Survey of storage systems for high-performance computing. Supercomput. Front. Innov. 5, 1 (2018), 3158.Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. [83] Ma Kwan-Liu. 2009. In situ visualization at extreme scale: Challenges and opportunities. IEEE Comput. Graph. Applic. 29, 6 (2009), 1419.Google ScholarGoogle ScholarDigital LibraryDigital Library
  84. [84] Machado Eduardo, Serqueira Marcello, Ogasawara Eduardo, Ogando Ricardo, Maia Marcio A. G., Costa Luiz Nicolaci da, Campisano Riccardo, Guedes Gustavo Paiva, and Bezerra Eduardo. 2016. Exploring machine learning methods for the star/galaxy separation problem. In International Joint Conference on Neural Networks (IJCNN). IEEE, New York, NY, 123130.Google ScholarGoogle ScholarCross RefCross Ref
  85. [85] Malewicz Grzegorz, Foster Ian, Rosenberg Arnold L., and Wilde Michael. 2007. A tool for prioritizing DAGMan jobs and its evaluation. J. Grid Comput. 5, 2 (2007), 197212.Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. [86] Meister Dirk, Kaiser Jurgen, Brinkmann Andre, Cortes Toni, Kuhn Michael, and Kunkel Julian. 2012. A study on data deduplication in HPC storage systems. In International Conference on High Performance Computing, Networking, Storage and Analysis. IEEE, New York, NY, 111.Google ScholarGoogle ScholarDigital LibraryDigital Library
  87. [87] Menon Harshitha, Bhatele Abhinav, Fourestier Sebastien, Kale Laxmikant, and Pellegrini Francois. 2015. Applying Graph Partitioning Methods in Measurement-based Dynamic Load Balancing. Technical Report, UIUC. https://www.ideals.illinois.edu/items/77157.Google ScholarGoogle Scholar
  88. [88] Myers Kary, Lawrence Earl, Fugate Michael, Bowen Claire McKay, Ticknor Lawrence, Woodring Jon, Wendelberger Joanne, and Ahrens Jim. 2016. Partitioning a large simulation as it runs. Technometrics 58, 3 (2016), 329340.Google ScholarGoogle ScholarCross RefCross Ref
  89. [89] Nielson Gregory M.. 2003. On marching cubes. IEEE Trans. Visualiz. Comput. Graph. 9, 3 (2003), 283297.Google ScholarGoogle ScholarDigital LibraryDigital Library
  90. [90] Oldfield Ron A., Widener Patrick, Maccabe Arthur B., Ward Lee, and Kordenbrock Todd. 2006. Efficient data-movement for lightweight I/O. In International Workshop on High Performance I/O Techniques and Deployment of Very Large Scale I/O Systems.Google ScholarGoogle Scholar
  91. [91] Ovsyannikov Andrey, Romanus Melissa, Straalen Brian Van, Weber Gunther H., and Trebotich David. 2016. Scientific workflows at DataWarp-speed: Accelerated data-intensive science using NERSC’s Burst Buffer. In 1st Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISCS). IEEE, New York, NY, 16.Google ScholarGoogle Scholar
  92. [92] Pallickara Sangmi Lee, Pallickara Shrideep, and Zupanski Milija. 2012. Towards efficient data search and subsetting of large-scale atmospheric datasets. Fut. Gen. Comput. Syst. 28, 1 (2012), 112118.Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. [93] Panasas 2022. Retrieved November 20, 2022 from https://www.panasas.com/.Google ScholarGoogle Scholar
  94. [94] Papadomanolakis Stratos, Ailamaki Anastassia, Lopez Julio C., Tu Tiankai, O’Hallaron David R., and Heber Gerd. 2006. Efficient query processing on unstructured tetrahedral meshes. In ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, 551562.Google ScholarGoogle ScholarDigital LibraryDigital Library
  95. [95] Post Frits H., Vrolijk Benjamin, Hauser Helwig, Laramee Robert S., and Doleisch Helmut. 2003. The state of the art in flow visualisation: Feature extraction and tracking. In Computer Graphics Forum, Vol. 22. Wiley Online Library, New York, NY, 775792.Google ScholarGoogle Scholar
  96. [96] Potter Douglas, Stadel Joachim, and Teyssier Romain. 2017. PKDGRAV3: Beyond trillion particle cosmological simulations for the next era of galaxy surveys. Computat. Astrophys. Cosmol. 4, 1 (2017), 113.Google ScholarGoogle Scholar
  97. [97] Prabhu Tarun and Gropp William. 2017. Moya–A JIT compiler for HPC. In Programming and Performance Visualization Tools. Springer, New York, NY, 5673.Google ScholarGoogle Scholar
  98. [98] Primack J. R., Dekel A., Koo D. C., Lapiner S., Ceverino D., Simons R. C., Snyder G. F., Bernardi M., Chen Z., Domínguez-Sánchez H., et al. 2018. Deep learning identifies High-z galaxies in a central blue nugget phase in a characteristic mass range. Astrophys. J. 858, 2 (2018), 114.Google ScholarGoogle ScholarCross RefCross Ref
  99. [99] Quintanas-Corominas Adrià, Maimí Pere, Casoni Eva, Turon Albert, Mayugo Joan Andreu, Guillamet Gerard, and Vázquez Mariano. 2018. A 3D transversally isotropic constitutive model for advanced composites implemented in a high performance computing code. Eur. J. Mechan.-A/Solids 71 (2018), 278291.Google ScholarGoogle ScholarCross RefCross Ref
  100. [100] Racah Evan, Beckham Christopher, Maharaj Tegan, Kahou Samira Ebrahimi, Prabhat, and Pal Christopher. 2017. ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. In 31st International Conference on Neural Information Processing Systems (NIPS’17). Curran Associates Inc., Red Hook, NY, 34053416. Google ScholarGoogle Scholar
  101. [101] Rew R., Hartnett E., Caron J., et al. 2006. netCDF-4: Software implementing an enhanced data model for the geosciences. In 22nd International Conference on Interactive Information Processing Systems for Meteorology, Oceanograph, and Hydrology.Google ScholarGoogle Scholar
  102. [102] Ross Robert, Ward Lee, Carns Philip, Grider Gary, Klasky Scott, Koziol Quincey, Lockwood Glenn K., Mohror Kathryn, Settlemyer Bradley, and Wolf Matthew. 2018. Storage Systems and I/O: Organizing, Storing, and Accessing Data for Scientific Discovery (Report for the DOE ASCR Workshop on Storage Systems and I/O). Technical Report. USDOE Office of Science (SC).Google ScholarGoogle Scholar
  103. [103] Roth Philip C.. 2007. Characterizing the I/O behavior of scientific applications on the Cray XT. In 2nd International Workshop on Petascale Data Storage at Supercomputing’07. 5055.Google ScholarGoogle Scholar
  104. [104] Rübel Oliver, Byna Surendra, Wu Kesheng, Li Fuyu, Wehner Michael, Bethel Wes, et al. 2012. TECA: A parallel toolkit for extreme climate analysis. Procedia Comput. Sci. 9 (2012), 866876.Google ScholarGoogle ScholarCross RefCross Ref
  105. [105] Rudyy Oleksandr, Garcia-Gasulla Marta, Mantovani Filippo, Santiago Alfonso, Sirvent Raül, and Vázquez Mariano. 2019. Containers in HPC: A scalability and portability study in production biological simulations. In IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, New York, NY, 567577.Google ScholarGoogle Scholar
  106. [106] Sevilla Michael A., Nasirigerdeh Reza, Maltzahn Carlos, LeFevre Jeff, Watkins Noah, Alvaro Peter, Lawson Margaret, Lofstead Jay, and Pivarski Jim. 2018. Tintenfisch: File system namespace schemas and generators. In 10th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage’18). USENIX Association, Berkeley, CA.Google ScholarGoogle Scholar
  107. [107] Silver Deborah and Zabusky Norman J.. 1993. Quantifying visualizations for reduced modeling in nonlinear science: Extracting structures from data sets. J. Vis. Commun. Image Represent. 4, 1 (1993), 4661.Google ScholarGoogle ScholarCross RefCross Ref
  108. [108] Sim Hyogi, Kim Youngjae, Vazhkudai Sudharshan S., Vallée Geoffroy R., Lim Seung-Hwan, and Butt Ali R.. 2017. TagIt: An integrated indexing and search service for file systems. In International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, New York, NY.Google ScholarGoogle ScholarDigital LibraryDigital Library
  109. [109] Skillman Samuel W., Warren Michael S., Turk Matthew J., Wechsler Risa H., Holz Daniel E., and Sutter P. M.. 2014. Dark Sky Simulations: Early Data Release. (2014). arxiv:astro-ph.CO/1407.2600.Google ScholarGoogle Scholar
  110. [110] SQLite 2022. SQLite. Retrieved from http://www.sqlite.org/.Google ScholarGoogle Scholar
  111. [111] Starfish 2017. Starfish. Retrieved from https://storageconference.us/2017/Presentations/Farmer.pdf.Google ScholarGoogle Scholar
  112. [112] Stockinger Kurt, Shalf John, Wu Kesheng, and Bethel E. Wes. 2005. Query-driven visualization of large data sets. In IEEE Visualization Conference. IEEE, New York, NY.Google ScholarGoogle Scholar
  113. [113] Su Yu, Agrawal Gagan, and Woodring Jonathan. 2012. Indexing and parallel query processing support for visualizing climate datasets. In 41st International Conference on Parallel Processing. IEEE, New York, NY, 249258.Google ScholarGoogle ScholarDigital LibraryDigital Library
  114. [114] Tang Houjun, Byna Suren, Dong Bin, Liu Jialin, and Koziol Quincey. 2017. SoMeta: Scalable object-centric metadata management for high performance computing. In IEEE International Conference on Cluster Computing (CLUSTER). IEEE, New York, NY, 359369.Google ScholarGoogle ScholarCross RefCross Ref
  115. [115] Tang Hong, Gulbeden Aziz, Zhou Jingyu, Strathearn William, Yang Tao, and Chu Lingkun. 2004. The Panasas ActiveScale storage cluster—delivering scalable high bandwidth storage. In ACM/IEEE Conference on Supercomputing. IEEE, New York, NY, 5353.Google ScholarGoogle ScholarDigital LibraryDigital Library
  116. [116] Tauheed Farhan, Biveinis Laurynas, Heinis Thomas, Schurmann Felix, Markram Henry, and Ailamaki Anastasia. 2012. Accelerating range queries for brain simulations. In IEEE 28th International Conference on Data Engineering. IEEE, New York, NY, 941952.Google ScholarGoogle ScholarDigital LibraryDigital Library
  117. [117] Thakar Ani R., Szalay Alex, Fekete George, and Gray Jim. 2008. The Catalog archive server database management system. Comput. Sci. Eng. 10, 1 (2008), 3037. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  118. [118] Thakur Rajeev, Gropp William, and Lusk Ewing. 1999. Data sieving and collective I/O in ROMIO. In 7th Symposium on the Frontiers of Massively Parallel Computation (Frontiers’99). IEEE, New York, NY, 182189.Google ScholarGoogle Scholar
  119. [119] Thompson David, Levine Joshua A., Bennett Janine C., Bremer Peer-Timo, Gyulassy Attila, Pascucci Valerio, and Pébay Philippe P.. 2011. Analysis of large-scale scalar data using hixels. In IEEE Symposium on Large Data Analysis and Visualization. IEEE, New York, NY, 2330.Google ScholarGoogle Scholar
  120. [120] Top500 2022. TOP500 Lists. Retrieved from https://www.top500.org/lists/top500/.Google ScholarGoogle Scholar
  121. [121] Tull Craig E., Essiari Abdelilah, Gunter Dan, et al. 2013. The SPOT Suite project. Retrieved from http://spot.nersc.gov/.Google ScholarGoogle Scholar
  122. [122] Tzeng Fan-Yin and Ma Kwan-Liu. 2005. Intelligent feature extraction and tracking for visualizing large-scale 4D flow simulations. In ACM/IEEE Conference on Supercomputing. IEEE, New York, NY.Google ScholarGoogle Scholar
  123. [123] Ullrich Paul A. and Zarzycki Colin M.. 2017. TempestExtremes: A framework for scale-insensitive pointwise feature tracking on unstructured grids. Geosci. Model Devel. 10, 3 (2017), 10691090.Google ScholarGoogle ScholarCross RefCross Ref
  124. [124] Ullrich Paul A., Zarzycki Colin M., McClenny Elizabeth E., Pinheiro Marielle C., Stansfield Alyssa M., and Reed Kevin A.. 2021. TempestExtremes v2.1: A community framework for feature detection, tracking and analysis in large datasets. Geosci. Model Devel. Discuss. 14 (2021), 5023–5048. https://gmd.copernicus.org/articles/14/5023/2021/.Google ScholarGoogle ScholarCross RefCross Ref
  125. [125] Ulmer Craig, Mukherjee Shyamali, Templet Gary, Levy Scott, Lofstead Jay, Widener Patrick, Kordenbrock Todd, and Lawson Margaret. 2018. Faodel: Data management for next-generation application workflows. In Workshop on Infrastructure for Workflows and Application Composition (IWAC).Google ScholarGoogle Scholar
  126. [126] Energy U.S. Department of. 2019. U.S. Department of Energy and Intel to Build First Exascale Supercomputer. Retrieved from https://www.energy.gov/articles/us-department-energy-and-intel-build-first-exascale-supercomputer.Google ScholarGoogle Scholar
  127. [127] VIC 2022. VIC. Retrieved from https://zenodo.org/badge/DOI/10.5281/zenodo.5781377.svg.Google ScholarGoogle Scholar
  128. [128] Wang Chaoli, Garcia Antonio, and Shen Han-Wei. 2006. Interactive level-of-detail selection using image-based quality metric for large volume visualization. IEEE Trans. Visualiz. Comput. Graph. 13, 1 (2006), 122134.Google ScholarGoogle ScholarDigital LibraryDigital Library
  129. [129] Wang Chaoli and Shen Han-Wei. 2006. LOD map—a visual interface for navigating multiresolution volume visualization. IEEE Trans. Visualiz. Comput. Graph. 12, 5 (2006), 10291036.Google ScholarGoogle ScholarDigital LibraryDigital Library
  130. [130] Wang Chaoli, Yu Hongfeng, and Ma Kwan-Liu. 2008. Importance-driven time-varying data visualization. IEEE Trans. Visualiz. Comput. Graph. 14, 6 (2008), 15471554.Google ScholarGoogle ScholarDigital LibraryDigital Library
  131. [131] Weil Sage A., Brandt Scott A., Miller Ethan L., Long Darrell D. E., and Maltzahn Carlos. 2006. Ceph: A scalable, high-performance distributed file system. In 7th Symposium on Operating Systems Design and Implementation. USENIX Association, Berkeley, CA, 307320.Google ScholarGoogle Scholar
  132. [132] Welch Brent and Noer Geoffrey. 2013. Optimizing a hybrid SSD/HDD HPC storage system based on file size distributions. In IEEE 29th Symposium on Mass Storage Systems and Technologies (MSST). IEEE, New York, NY, 112.Google ScholarGoogle Scholar
  133. [133] Welch Brent, Unangst Marc, Abbasi Zainul, Gibson Garth A., Mueller Brian, Small Jason, Zelenka Jim, and Zhou Bin. 2008. Scalable performance of the Panasas parallel file system. In USENIX Conference on File and Storage Technologies. 117.Google ScholarGoogle Scholar
  134. [134] Wendelberger Joanne Roth, Banesh Divya, Wendelberger Laura Jean, and Ahrens James Paul. 2017. Detecting Changes in Simulations. Technical Report. Los Alamos National Lab. (LANL), Los Alamos, NM.Google ScholarGoogle Scholar
  135. [135] Widanagamaachchi Wathsala, Christensen Cameron, Pascucci Valerio, and Bremer Peer-Timo. 2012. Interactive exploration of large-scale time-varying data using dynamic tracking graphs. In IEEE Symposium on Large Data Analysis and Visualization (LDAV). IEEE, New York, NY, 917.Google ScholarGoogle ScholarCross RefCross Ref
  136. [136] Wu K., Ahern S., Bethel E. W., Chen J., Childs H., Geddes C., Gu J., Hagen H., Hamann B., Lauret J., Meredith J., Messmer P., Otoo E., Poskanzer A., Rübel O., Shoshani A., Sim A., Stockinger K., Weber G., Zhang W. M., et al. 2009. FastBit: Interactively searching massive data. In Scientific Discovery through Advanced Computing Conference.Google ScholarGoogle ScholarCross RefCross Ref
  137. [137] Wu Tzuhsien, Chou Jerry, Hao Shyng, Dong Bin, Klasky Scott, and Wu Kesheng. 2017. Optimizing the query performance of block index through data analysis and I/O modeling. In International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, New York, NY.Google ScholarGoogle ScholarDigital LibraryDigital Library
  138. [138] Yu Hongfeng, Wang Chaoli, Grout Ray W., Chen Jacqueline H., and Ma Kwan-Liu. 2010. In situ visualization for large-scale combustion simulations. IEEE Comput. Graph. Applic. 30, 3 (2010), 4557.Google ScholarGoogle ScholarDigital LibraryDigital Library
  139. [139] Zaspel Peter and Griebel Michael. 2011. Massively parallel fluid simulations on Amazon’s HPC cloud. In 1st International Symposium on Network Cloud Computing and Applications. IEEE, New York, NY, 7378.Google ScholarGoogle Scholar
  140. [140] Zhang Wei, Byna Suren, Tang Houjun, Williams Brody, and Chen Yong. 2019. MIQS: Metadata indexing and querying service for self-describing file formats. In International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, New York, NY, 5.Google ScholarGoogle ScholarDigital LibraryDigital Library
  141. [141] Zhao Ming, Held Isaac M., Lin Shian-Jiann, and Vecchi Gabriel A.. 2009. Simulations of global hurricane climatology, interannual variability, and response to global warming using a 50-km resolution GCM. J. Clim. 22, 24 (2009), 66536678.Google ScholarGoogle ScholarCross RefCross Ref
  142. [142] Zheng Qing, Cranor Charles D., Guo Danhao, Ganger Gregory R., Amvrosiadis George, Gibson Garth A., Settlemyer Bradley W., Grider Gary, and Guo Fan. 2018. Scaling embedded in-situ indexing with DeltaFS. In International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, New York, NY, 3044.Google ScholarGoogle ScholarDigital LibraryDigital Library
  143. [143] Zheng Qing, Ren Kai, Gibson Garth, Settlemyer Bradley W., and Grider Gary. 2015. DeltaFS: Exascale file systems scale better without dedicated servers. In 10th Parallel Data Storage Workshop (PDSW’15). ACM, New York, NY, 16. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  144. [144] Zhou Bo and Chiang Yi-Jen. 2018. Key time steps selection for large-scale time-varying volume datasets using an information-theoretic storyboard. In Computer Graphics Forum, Vol. 37. Wiley Online Library, New York, NY, 3749.Google ScholarGoogle Scholar

Index Terms

  1. EMPRESS: Accelerating Scientific Discovery through Descriptive Metadata Management

                Recommendations

                Comments

                Login options

                Check if you have access through your login credentials or your institution to get full access on this article.

                Sign in

                Full Access

                • Published in

                  cover image ACM Transactions on Storage
                  ACM Transactions on Storage  Volume 18, Issue 4
                  November 2022
                  279 pages
                  ISSN:1553-3077
                  EISSN:1553-3093
                  DOI:10.1145/3570642
                  Issue’s Table of Contents

                  Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

                  Publisher

                  Association for Computing Machinery

                  New York, NY, United States

                  Publication History

                  • Published: 12 December 2022
                  • Online AM: 27 September 2022
                  • Accepted: 2 March 2022
                  • Revised: 17 January 2022
                  • Received: 6 January 2021
                  Published in tos Volume 18, Issue 4

                  Permissions

                  Request permissions about this article.

                  Request Permissions

                  Check for updates

                  Qualifiers

                  • research-article
                  • Refereed
                • Article Metrics

                  • Downloads (Last 12 months)218
                  • Downloads (Last 6 weeks)14

                  Other Metrics

                PDF Format

                View or Download as a PDF file.

                PDF

                eReader

                View online with eReader.

                eReader

                Full Text

                View this article in Full Text.

                View Full Text

                HTML Format

                View this article in HTML Format .

                View HTML Format
                About Cookies On This Site

                We use cookies to ensure that we give you the best experience on our website.

                Learn more

                Got it!