Abstract
Enterprise and cloud data centers are comprised of tens of thousands of servers providing petabytes of storage to a large number of users and applications. At such a scale, these storage systems face two key challenges: (1) hot-spots due to the dynamic popularity of stored objects; and (2) high operational costs due to power and cooling. Existing storage solutions, however, are unsuitable to address these challenges because of the large number of servers and data objects. This article describes the design, implementation, and evaluation of Ursa, a system that scales to a large number of storage nodes and objects, and aims to minimize latency and bandwidth costs during system reconfiguration. Toward this goal, Ursa formulates an optimization problem that selects a subset of objects from hot-spot servers and performs topology-aware migration to minimize reconfiguration costs. As exact optimization is computationally expensive, we devise scalable approximation techniques for node selection and efficient divide-and-conquer computation. We also show that the same dynamic reconfiguration techniques can be leveraged to reduce power costs by dynamically migrating data off under-utilized nodes, and powering up servers neighboring existing hot-spots to reduce reconfiguration costs. Our evaluation shows that Ursa achieves cost-effective load management, is time-responsive in computing placement decisions (e.g., about two minutes for 10K nodes and 10M objects), and provides power savings of 15%--37%.
- Abd-El-Malek, M., Ii, W. V. C., Cranor, C., Ganger, G. R., Hendricks, J., Klosterman, A. J., Mesnier, M., Prasad, M., Salmon, B., Sambasivan, R. R., Sinnamohideen, S., Strunk, J. D., Thereska, E., Wachs, M., and Wylie, J. J. 2005. Ursa Minor: Versatile cluster-based storage. In Proceedings of the FAST Conference. Google Scholar
Digital Library
- Amazon S3. 2012. http://aws.amazon.com/s3/.Google Scholar
- Barroso, L. A. and Ölzle, U. 2007. The case for energy-proportional computing. Computer 40, 33--37. Google Scholar
Digital Library
- Chang, F., Dean, J., Ghemawat, S., Hsieh, W. C., Wallach, D. A., Burrows, M., Chandra, T., Fikes, A., and Gruber, R. E. 2006. Bigtable: A distributed storage system for structured data. In Proceedings of the OSDI Conference. Google Scholar
Digital Library
- Chase, J. S., Anderson, D. C., Thakar, P. N., Vahdat, A. M., and Doyle, R. P. 2001. Managing energy and server resources in hosting centers. In Proceedings of the SOSP Conference. Google Scholar
Digital Library
- Curino, C., Jones, E., Zhang, Y., and Madden, S. 2010. Schism: A workload-driven approach to database replication and partitioning. In Proceedings of the VLDB Conference. Google Scholar
Digital Library
- Curino, C., Jones, E., Popa, R. A., Malviya, N., Wu, E., Madden, S., Balakrishnan, H., and Zeldovich, N. 2011. Relational Cloud: A database service for the Cloud. In Proceedings of the CIDR Conference.Google Scholar
- Das, S., Nishimura, S., Agrawal, D., and Abbadi, A. E. 2011. Albatross: Lightweight elasticity in shared storage databases for the Cloud using live data migration. In Proceedings of the VLDB Conference. Google Scholar
Digital Library
- Dean, J. and Ghemawat, S. 2004. MapReduce: Simplified data processing on large clusters. In Proceedings of the OSDI Conference. Google Scholar
Digital Library
- Elmore, A., Das, S., Agrawal, D., and Abbadi, A. E. 2010. Who’s driving this Cloud? Towards efficient migration for elastic and autonomic multitenant databases. Tech. rep., University of California, Santa Barbara.Google Scholar
- Elmore, A., Das, S., Agrawal, D., and Abbadi, A. E. 2011. Zephyr: Live migration in shared nothing databases for elastic Cloud platforms. In Proceedings of the SIGMOD Conference. Google Scholar
Digital Library
- Eric, E. A., Spence, S., Swaminathan, R., Kallahalla, M., and Wang, Q. 2005. Quickly finding near-optimal storage designs. ACM Trans. Comput. Syst. 23, 337--374. Google Scholar
Digital Library
- Ganesh, L., Weatherspoon, H., Balakrishnan, M., and Birman, K. 2007. Optimizing power consumption in large scale storage systems. In Proceedings of the HOTOS Conference. Google Scholar
Digital Library
- Ghemawat, S., Gobioff, H., and Leung, S.-T. 2003. The Google file system. SIGOPS Oper. Syst. Rev. 37, 29--43. Google Scholar
Digital Library
- Greenberg, A. G., Hamilton, J. R., Jain, N., Kandula, S., Kim, C., Lahiri, P., Maltz, D. A., Patel, P., and Sengupta, S. 2009. VL2: A scalable and flexible data center network. In Proceedings of the SIGCOMM Conference. Google Scholar
Digital Library
- Gulati, A., Kumar, C., Ahmad, I., and Kumar, K. 2010. BASIL: Automated IO load balancing across storage devices. In Proceedings of the FAST Conference. Google Scholar
Digital Library
- Hiller, F. S. and Lieberman, G. J. 2005. Introduction to Operations Research 8th Ed., McGraw-Hill. Google Scholar
Digital Library
- Kunkle, D. and Schindler, J. 2008. A load balancing framework for clustered storage systems. In Proceedings of the HiPC Conference. Google Scholar
Digital Library
- Lang, W., Patel, J. M., and Naughton, J. F. 2010. On energy management, load balancing and replication. SIGMOD Rec. 38, 35--42. Google Scholar
Digital Library
- Lim, H. C., Babu, S., and Chase, J. S. 2010. Automated control for elastic storage. In Proceedings of the ICAC Conference. Google Scholar
Digital Library
- Litwin, W. 1980. Linear hashing: A new tool for file and table addressing. In Proceedings of the VLDB Conference. Google Scholar
Digital Library
- Narayanan, D., Donnelly, A., and Rowstron, A. 2008. Write off-loading: Practical power management for enterprise storage. ACM Trans. Storage, 10:1--10:23. Google Scholar
Digital Library
- Narayanan, D., Donnelly, A., Thereska, E., Elnikety, S., and Rowstron, A. 2008. Everest: Scaling down peak loads through I/O off-loading. In Proceedings of the OSDI Conference. Google Scholar
Digital Library
- Sankar, S., Gurumurthi, S., and Stan, M. R. 2008. Sensitivity-based power management of enterprise storage systems. In Proceedings of the MASCOTS Conference.Google Scholar
- Savinov, S. and Daudjee, K. 2010. Dynamic database replica provisioning through virtualization. In Proceedings of the CloudDB Conference. Google Scholar
Digital Library
- Tam, H. V., Chen, C., and Ooi, B. C. 2010. Towards elastic transactional cloud storage with range query support. In Proceedings of the VLDB Conference.Google Scholar
- Thereska, E., Donnelly, A., and Narayanan, C. 2009. Sierra: A power-proportional, distributed storage system. Tech. rep. MSR-TR-2009-153.Google Scholar
- Tsirogiannis, D., Harizopoulos, S., and Shah, M. A. 2010. Analyzing the energy efficiency of a database server. In Proceedings of the SIGMOD Conference. Google Scholar
Digital Library
- Venkataramani, A., Kokku, R., and Dahlin, M. 2002. TCP Nice: A mechanism for background transfers. In Proceedings of the OSD Conference. Google Scholar
Digital Library
- Verma, A., Ahuja, P., and Neogi, A. 2008. pMapper: Power and migration cost aware application placement in virtualized systems. In Proceedings of the Middleware Conference. Google Scholar
Digital Library
- Verma, A., Koller, R., Useche, L., and Rangaswami, R. 2010. SRCMap: Energy proportional storage using dynamic consolidation. In Proceedings of the FAST Conference. Google Scholar
Digital Library
- Wang, Z., Zhu, X., Singhal, S., and Packard, H. 2005. Utilization and SLO-based control for dynamic sizing of resource partitions. In Proceedings of the DSOM Conference. Google Scholar
Digital Library
- Weil, S. A., Brand, S. A., Miller, E. L., Long, D. D. E., and Maltzahn, C. 2006. Ceph: A scalable, high-performance distributed file system. In Proceedings of the OSDI Conference. Google Scholar
Digital Library
- Windows Azure. 2012. http://www.microsoft.com/windowsazure/.Google Scholar
- Xu, Z., Tu, Y.-C., and Wang, X. 2010. Exploring power-performance tradeoffs in database systems. In Proceedings of the ICDE Conference.Google Scholar
- Yin, Q., Schüpbach, A., Cappos, J., Baumann, A., and Roscoe, T. 2009. Rhizoma: A runtime for self-deploying, self-managing overlays. In Proceedings of the Middleware Conference. Google Scholar
Digital Library
- You, G., Hwang, S., Jain, N., and Zeng, H.-J. 2011. Scalable load balancing in cluster storage systems. In Proceedings of the Middleware Conference. Google Scholar
Digital Library
- Zeng, L., Feng, D., Wang, F., and Zhou, K. 2005. A strategy of load balancing in objects storage system. In Proceedings of the CIT Conference. Google Scholar
Digital Library
Index Terms
Ursa: Scalable Load and Power Management in Cloud Storage Systems
Recommendations
Scalable load balancing in cluster storage systems
Middleware'11: Proceedings of the 12th ACM/IFIP/USENIX international conference on MiddlewareEnterprise and cloud data centers are comprised of tens of thousands of servers providing petabytes of storage to a large number of users and applications. At such a scale, these storage systems face two key challenges: (a) hot-spots due to the dynamic ...
Scalable load balancing in cluster storage systems
Middleware '11: Proceedings of the 12th International Middleware ConferenceEnterprise and cloud data centers are comprised of tens of thousands of servers providing petabytes of storage to a large number of users and applications. At such a scale, these storage systems face two key challenges: (a) hot-spots due to the dynamic ...
Using Working Set Reorganization to Manage Storage Systems with Hard and Solid State Disks
ICPPW '14: Proceedings of the 2014 43rd International Conference on Parallel Processing WorkshopsScientific applications from many problem domains produce and/or access large volumes of data. To support these applications, designers of high-end computing (HEC) systems have greatly increased the capacity of storage systems in recent years. However, ...






Comments