Abstract
Specialization of datacenter resources brings performance and energy improvements in response to the growing scale and diversity of cloud applications. Yet heterogeneous hardware adds complexity and volatility to latency-sensitive applications. A resource allocation mechanism that leverages architectural principles can overcome both of these obstacles.
We integrate research in heterogeneous architectures with recent advances in multi-agent systems. Embedding architectural insight into proxies that bid on behalf of applications, a market effectively allocates hardware to applications with diverse preferences and valuations. Exploring a space of heterogeneous datacenter configurations, which mix server-class Xeon and mobile-class Atom processors, we find an optimal heterogeneous balance that improves both welfare and energy-efficiency. We further design and evaluate twelve design points along the Xeon-to-Atom spectrum, and find that a mix of three processor architectures achieves a 12× reduction in response time violations relative to equal-power homogeneous systems.
- Yuvraj Agarwal, Steve Hodges, Ranveer Chandra, James Scott, Paramvir Bahl, and Rajesh Gupta. 2009. Somniloquy: Augmenting network interfaces to reduce PC energy usage. In Proceedings of the 6th Symposium on Networked Systems Design and Implementation (NSDI). USENIX Association, Berkeley, CA, 365--380. Google Scholar
Digital Library
- Amazon. 2009. Elastic cloud computing. http://aws.amazon.com/ec2/.Google Scholar
- David G. Andersen, Jason Franklin, Michael Kaminsky, Amar Phanishayee, Lawrence Tan, and Vijay Vasudevan. 2009. FAWN: A fast array of wimpy nodes. In Proceedings of the 22nd Symposium on Operating Systems Principles (SOSP). ACM, New York, 1--14. Google Scholar
Digital Library
- Anonymous. 2012. Space Invaders. The Economist.Google Scholar
- Alvin Auyoung, Brent N. Chun, Alex C. Snoeren, and Amin Vahdat. 2004. Resource allocation in federated distributed computing infrastructures. In Proceedings of the 1st Workshop on Operating System and Architectural Support for the On-Demand IT Infrastructure. 1--10.Google Scholar
- Luiz André Barroso, Kourosh Gharachorloo, Robert McNamara, Andreas Nowatzyk, Shaz Qadeer, Barton Sano, Scott Smith, Robert Stets, and Ben Verghese. 2000. Piranha: A scalable architecture based on single-chip multiprocessing. In Proceedings of the 27th International Symposium on Computer Architecture (ISCA). ACM, New York, 282--293. Google Scholar
Digital Library
- Luiz André Barroso and Urs Hölzle. 2007. The case for energy-proportional computing. IEEE Comput. 40, 12, 33--37. Google Scholar
Digital Library
- Luiz André Barroso and Urs Hölzle. 2009. The datacenter as a computer. In Synthesis Lectures on Computer Architecture.Google Scholar
- Nathan Binkert, Bradford Beckmann, Gabriel Black, Steven K. Reinhardt, Ali Saidi, Arkaprava Basu, Joel Hestness, Derek R. Hower, Tushar Krishna, Somayeh Sardashti, Rathijit Sen, Korey Sewell, Muhammad Shoaib, Nilay Vaish, Mark D. Hill, and David A. Wood. 2011. The gem5 simulator. SIGARCH Comput. Archit. News 39, 2, 1--7. Google Scholar
Digital Library
- James Broberg, Srikumar Venugopal, and Rajkumar Buyya. 2007. Market-oriented grids and utility computing: The state-of-the-art and future directions. J. Grid Comput. 6, 3, 255--270.Google Scholar
Cross Ref
- Andrew Byde. 2002. Applying evolutionary game theory to auction mechanism design. In Proceedings of the 4th ACM Conference on Electronic Commerce. ACM, New York, 192--193. Google Scholar
Digital Library
- Andrew Byde. 2006. A comparison between mechanisms for sequential compute resource auctions. In Proceedings of the 5th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS). ACM, New York, 1199--1201. Google Scholar
Digital Library
- Andrew Byde, Mathias Sallé, and Claudio Bartolini. 2003. Market-based resource allocation for utility data centers. Tech. Rep.Google Scholar
- Rodrigo N. Calhieros, Rajid Ranjan, Anton Beloglazov, César A. F. De Rose, and Rajkumar Buyya. 2011. CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw.: Practice and Exper. 41, 23--50. Google Scholar
Digital Library
- Jeffrey S. Chase, Darrell C. Anderson, Prachi N. Thakar, Amin M. Vahdat, and Ronald P. Doyle. 2001. Managing energy and server resources in hosting centers. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP). ACM, New York, 103--116. Google Scholar
Digital Library
- Niket K. Choudhary, Salil V. Wadhavkar, Tanmay A. Shah, Hiran Mayukh, Jayneel Gandhi, Brandon H. Dwiel, Sandeep Navada, Hashem H. Najaf-abadi, and Eric Rotenberg. 2011. FabScalar: Composing synthesizable RTL designs of arbitrary cores within a canonical superscalar template. In Proceedings of the 38th International Symposium on Computer Architecture (ISCA). ACM, New York, 11--22. Google Scholar
Digital Library
- Rachel Courtland. 2012. The battle between ARM and Intel gets real. IEEE Spectrum.Google Scholar
- John D. Davis, James Laudon, and Kunle Olukotun. 2005. Maximizing CMP throughput with mediocre cores. In Proceedings of the 14th International Conference on Parallel Architectures and Compilation Techniques (PACT). IEEE Computer Society, Los Alamitos, CA, 51--62. Google Scholar
Digital Library
- Lieven Eeckhout, Sebastien Nussbaum, James E. Smith, and Koen DeBosschere. 2003. Statistical simulation: Adding efficiency to the computer designer's toolbox. IEEE Micro 23, 5, 26--38. Facebook. 2011. More effective computing. Tech. Rep. Google Scholar
Digital Library
- Michael Ferdman, Almutaz Adileh, Onur Kocberber, Stavros Volos, Mohammad Alisafaee, Djordje Jevdjic, Cansu Kaynak, Adrian Daniel Popescu, Anastasia Ailamaki, and Babak Falsafi. 2012. Clearing the clouds: A study of emerging scale-out workloads on modern hardware. In Proceedings of the 17th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, New York, 37--48. Google Scholar
Digital Library
- Donald F. Ferguson, Christos Nikolaou, Jakka Sairamesh, and Yechiam Yemini. 1996. Economic models for allocating resources in computer systems. In Market-Based Control, World Scientific Publishing Co., Inc., River Edge, NJ, 156--183. Google Scholar
Digital Library
- Anshul Gandhi, Mor Harchol-Balter, and Michael A. Kozuch. 2011. The case for sleep states in servers. In Proceedings of the 4th Workshop on Power-Aware Computing and Systems. ACM, New York, 2:1--2:5. Google Scholar
Digital Library
- Siddharth Garg, Shreyas Sundaram, and Hiren D. Patel. 2011. Robust heterogeneous data center design: A principled approach. SIGMETRICS Perform. Eval. Rev. 39, 3, 28--30. Google Scholar
Digital Library
- Varghese George, Sanjeev Jahagirdar, Chao Tong, K. Smits, Satish Damaraju, Scott Siers, Ves Naydenov, Tanveer Khondker, Sanjib Sarkar, and Puneet Singh. 2007. Penryn: 45-nm next generation intel core 2 processor. In Proceedings of the Asian Solid-State Circuits Conference (ASSCC). IEEE, Los Alamitos, CA, 14--17.Google Scholar
- Gianfranco Gerosa, Steve Curtis, Micahel D'Addeo, Bo Jiang, Belliappa Kuttanna, Feroze Merchant, Bina Patel, Mohammed Taufique, and Haytham Samarchi. 2009. A sub-2 W low power IA processor for mobile internet devices in 45 nm high-k metal gate CMOS. IEEE J. Solid-State Circ. 44, 1, 73--82.Google Scholar
Cross Ref
- Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andy Konwinski, Scott Shenker, and Ion Stoica. 2011. Dominant resource fairness: Fair allocation of multiple resource types. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation (NSDI). USENIX Association, Berkeley, CA, 24. Google Scholar
Digital Library
- Susan L. Graham, Peter B. Kessler, and Marshall K. Mckusick. 1982. Gprof: A call graph execution profiler. In Proceedings of the SIGPLAN Symposium on Compiler Construction (CC). ACM, New York, 120--126. Google Scholar
Digital Library
- Boris Grot, Damien Hardy, Pejman Lotfi-Kamran, and Babak Falsafi. 2012. Optimizing datacenter TCO with scale-out processors. IEEE Micro 32, 5, 52--63. Google Scholar
Digital Library
- Brian Guenter, Navendu Jain, and Charles Williams. 2011. Managing cost, performance, and reliability tradeoffs for energy-aware server provisioning. In Proceedings of the 30th International Conference on Computer Communications (INFOCOM). 1332--1340.Google Scholar
Cross Ref
- Marisabel Guevara, Benjamin Lubin, and Benjamin C. Lee. 2013. Navigating heterogeneous processors with market mechanisms. In Proceedings of the 19th International Symposium on High Performance Computer Architecture (HPCA). IEEE, Los Alamitos, CA, 95--106. Google Scholar
Digital Library
- Varun Gupta, Mor Harchol-Balter, J. G. Dai, and B. Zwart. 2010. On the inapproximability of M/G/k. Queue. Syst. Theory Appl. 64, 1, 5--48. Google Scholar
Digital Library
- Mark Hill and Michael Marty. 2008. Amdahl's Law in the multi-core era. IEEE Computer 41, 7, 33--38. Google Scholar
Digital Library
- Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, and Ion Stoica. 2011. Mesos: A platform for fine-grained resource sharing in the data center. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation (NSDI). USENIX Association, Berkeley, CA, 22. Google Scholar
Digital Library
- Mark Horowitz, Elad Alon, Dinesh Patil, Samuel Naffziger, Rajesh Kumar, and Kerry Bernstein. 2005. Scaling, power, and the future of CMOS. In International Electron Devices Meeting Technical Digest (IEDM). IEEE, Los Alamitos, CA, 7--15.Google Scholar
Cross Ref
- Toshihide Ibaraki and Naoki Katoh. 1988. Resource allocation problems: Algorithmic Approaches. Vol. 45, MIT Press, Cambridge, MA. Google Scholar
Digital Library
- Intel. 2009. VTune. http://software.intel.com/en-us/intel-vtune. Intel. 2011. Intel 64 and IA-32 Architectures Software Developers Manual. Intel.Google Scholar
- Vijay Janapa Reddi, Benjamin C. Lee, Trishul Chilimbi, and Kushagra Vaid. 2010. Web search using mobile cores: quantifying and mitigating the price of efficiency. In Proceedings of the 37th International Symposium on Computer Architecture (ISCA). ACM, New York, 314--325. Google Scholar
Digital Library
- Laura Keys, Suzanne Rivoire, and John D. Davis. 2012. The search for energy-efficient building blocks for the data center. In Proceedings of the International Conference on Computer Architecture. Springer-Verlag, Berlin, 172--182. Google Scholar
Digital Library
- Poonacha Kongetira and Kathirgamar Aingaran. 2005. Niagara: A 32-way multithreaded sparc processor. IEEE Micro 25, 2, 21--29. Google Scholar
Digital Library
- Rakesh Kumar, Keith I. Farkas, Norman P. Jouppi, Parthasarathy Ranganathan, and Dean M. Tullsen. 2003. Single-ISA heterogeneous multi-core architectures: The potential for processor power reduction. In Proceedings of the 36th International Symposium on Microarchitecture (MICRO). IEEE Computer Society, Los Alamitos, CA, 81. Google Scholar
Digital Library
- Rakesh Kumar, Dean M. Tullsen, and Norman P. Jouppi. 2006. Core architecture optimization for heterogeneous chip multiprocessors. In Proceedings of the 15th International Conference on Parallel Architectures and Compilation Techniques (PACT). ACM, New York, 23--32. Google Scholar
Digital Library
- Kevin Lai, Lars Rasmusson, Eytan Adar, Li Zhang, and Bernardo A. Huberman. 2005. Tycoon: An implementation of a distributed, market-based resource allocation system. Multiagent Grid Syst. 1, 3, 169--182. Google Scholar
Digital Library
- Benjamin C. Lee and David M. Brooks. 2006. Accurate and efficient regression modeling for microarchitectural performance and power prediction. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, New York, 185--194. Google Scholar
Digital Library
- Benjamin C. Lee and David M. Brooks. 2007. In Proceedings of the 13th International Symposium on High Performance Computer Architecture (HPCA). IEEE, Los Alamitos, CA, 340--351. Google Scholar
Digital Library
- Gunho Lee, Byung-Gon Chun, and H. Katz. 2011. Heterogeneity-aware resource allocation and scheduling in the cloud. In Proceedings of the 3rd USENIX Conference on Hot Topics in Cloud Computing. USENIX Association, Berkeley, CA, 4--4. Google Scholar
Digital Library
- Sheng Li, Jung Ho Ahn, Richard D. Strong, Jay B. Brockman, Dean M. Tullsen, and Norman P. Jouppi. 2009. McPAT: An integrated power, area, and timing modeling framework for multicore and manycore architectures. In Proceedings of the 42nd International Symposium on Microarchitecture (MICRO). ACM, New York, 469--480. Google Scholar
Digital Library
- Sheng Li, Kevin Lim, Paolo Faraboschi, Jichuan Chang, Parthasarathy Ranganathan, and Norman P. Jouppi. 2011. System-level integrated server architectures for scale-out datacenters. In Proceedings of the 44th International Symposium on Microarchitecture (MICRO). ACM, New York, 260--271. Google Scholar
Digital Library
- Kevin Lim, Parthasarathy Ranganathan, Jichuan Chang, Chandrakant Patel, Trevor Mudge, and Steven Reinhardt. 2008. Understanding and designing new server architectures for emerging warehouse-computing environments. In Proceedings of the 35th International Symposium on Computer Architecture (ISCA). IEEE Computer Society, Los Almitos, CA, 315--326. Google Scholar
Digital Library
- Pejman Lotfi-Kamran, Boris Grot, Michael Ferdman, Stavros Volos, Onur Kocberber, Javier Picorel, Almutaz Adileh, Djordje Jevdjic, Sachin Idgunji, Emre Ozer, and Babak Falsafi. 2012. Scale-out processors. In Proceedings of the 39th International Symposium on Computer Architecture (ISCA). IEEE Computer Society, Los Alamitos, CA, 500--511. Google Scholar
Digital Library
- Benjamin Lubin, Jeffrey O. Kephart, Rajarshi Das, and David C. Parkes. 2009. Expressive power-based resource allocation for data centers. In Proceedings of the 21st International Joint Conference on Artifical Intelligence (IJCAI). Morgan-Kaufmann Publishers Inc., San Francisco, CA, 1451--1456. Google Scholar
Digital Library
- Krishna T. Malladi, Benjamin C. Lee, Frank A. Nothaft, Christos Kozyrakis, Karthika Periyathambi, and Mark Horowitz. 2012. Towards energy-proportional datacenter memory with mobile DRAM. In Proceedings of the 39th International Symposium on Computer Architecture (ISCA). IEEE Computer Society, Los Alamitos, CA, 37--48. Google Scholar
Digital Library
- Jason Mars, Lingjia Tang, and Robert Hundt. 2011. Heterogeneity in Homogeneous Warehouse-Scale Computers: A Performance Opportunity. IEEE Comput. Archit. Lett. 10, 2, 29--32. Google Scholar
Digital Library
- David Meisner, Brian T. Gold, and Thomas F. Wenisch. 2009. PowerNap: Eliminating server idle power. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, New York, 205--216. Google Scholar
Digital Library
- David Meisner, Christopher M. Sadler, Luiz André Barroso, Wolf-Dietrich Weber, and Thomas F. Wenisch. 2011. Power management of online data-intensive services. In Proceedings of the 38th International Symposium on Computer Architecture (ISCA). ACM, New York, 319--330. Google Scholar
Digital Library
- David Meisner and Thomas F Wenisch. 2010. Stochastic queuing simulation for data center workloads. In Proceedings of the Workshop on Energy-Efficient Design.Google Scholar
- Jeffrey Mogul, Jayaram Mudigonda, Nathan Binkert, Parthasarathy Ranganathan, and Vanish Talwar. 2008. Using asymmetric single-ISA CMPs to save energy on operating systems. IEEE Computer 28, 3, 26--41. Google Scholar
Digital Library
- Moor Insights and Strategy. 2013. HP Moonshot: An accelerator for hyperscale workloads. Tech. Rep.Google Scholar
- Ripal Nathuji, Canturk Isci, and Eugene Gorbatov. 2007. Exploiting platform heterogeneity for power efficient data centers. In Proceedings of the 4th International Conference on Autonomous Computing (ICAC). IEEE, Los Alamitos, CA, 5. Google Scholar
Digital Library
- Open Source. 2010. OProfile. http://oprofile.sourceforge.net.Google Scholar
- John Ousterhout, Parag Agrawal, David Erickson, Christos Kozyrakis, Jacob Leverich, David Mazières, Subhasish Mitra, Aravind Narayanan, Guru Parulkar, Mendel Rosenblum, Stephen M. Rumble, Eric Stratmann, and Ryan Stutsman. 2010. The case for RAMClouds: Scalable high-performance storage entirely in DRAM. SIGOPS Oper. Syst. Rev. 43, 4, 92--105. Google Scholar
Digital Library
- David C. Parkes, Ariel D. Procaccia, and Nisarg Shah. 2012. Beyond dominant resource fairness: Extensions, limitations, and indivisibilities. In Proceedings of the 13th Conference on Electronic Commerce (EC). ACM, New York, 808--825. Google Scholar
Digital Library
- Aashish Phansalkar, Ajay Joshi, and Lizy K. John. 2007. Analysis of redundancy and application balance in the SPEC CPU2006 benchmark suite. In Proceedings of the 34th International Symposium on Computer Architecture (ISCA). ACM, New York, 412--423. Google Scholar
Digital Library
- Asfandyar Qureshi, Rick Weber, Hari Balakrishnan, John Guttag, and Bruce Maggs. 2009. Cutting the electric bill for internet-scale systems. In Proceedings of the ACM SIGCOMM 2009 Conference on Data Communication (SIGCOMM). ACM, New York, 123--134. Google Scholar
Digital Library
- Gang Ren, Eric Tune, Tipp Moseley, Yixin Shi, Silvius Rus, and Robert Hundt. 2010. Google-wide profiling: A continuous profiling infrastructure for data centers. IEEE Micro 30, 4, 65--79. Google Scholar
Digital Library
- Cosmin Rusu, Alexandre Ferreira, Claudio Scordino, and Aaron Watson. 2006. Energy-efficient real-time heterogeneous server clusters. In Proceedings of the 12th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS). IEEE Computer Society, Los Alamitos, CA, 418--428. Google Scholar
Digital Library
- Seamicro. 2011. SeaMicro Introduces the SM10000-64HD.Google Scholar
- Sena Seneviratne and David C Levy. 2010. Cost profile prediction for grid computing. Concurr. Computat. Practice Experi. 22, 1 107--142. Google Scholar
Digital Library
- Michael Stonebraker, Paul M. Aoki, Witold Litwin, Avi Pfeffer, Adam Sah, Jeff Sidell, Carl Staelin, and Andrew Yu. 1996. Mariposa: A wide-area distributed database system. VLDB J. 5, 1, 048--063. Google Scholar
Digital Library
- M. Aater Suleman, Onur Mutlu, Moinuddin K. Qureshi, and Yale N. Patt. 2009. Accelerating critical section execution with asymmetric multi-core architectures. In Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, New York, 253--264. Google Scholar
Digital Library
- Ivan E. Sutherland. 1968. A futures market in computer time. Commun. ACM 11, 6, 449--451. Google Scholar
Digital Library
- U.S. Environmental Protection Agency. 2007. Report to Congress on Server and Data Center Energy Efficiency.Google Scholar
- Christian Vecchiola, Rodrigo N. Calheiros, Dileban Karunamoorthy, and Rajkumar Buyya. 2012. Deadline-driven provisioning of resources for scientific applications in hybrid clouds with Aneka. Future Gen. Comput. Syst. 28, 1, 58--65. Google Scholar
Digital Library
- Carl A. Waldspurger, Tad Hogg, Bernardo A. Huberman, Jeffrey O. Kephart, and W. Scott Stornetta. 1992. Spawn: A Distributed Computational Economy. IEEE Trans. Softw. Eng. 18, 2, 103--117. Google Scholar
Digital Library
- Weidan Wu and Benjamin C. Lee. 2012. Inferred Models for Dynamic and Sparse Hardware-Software Spaces. In Proceedings of the 45th International Symposium on Microarchitecture (MICRO). IEEE Computer Society, Los Alamitos, CA, 413--424. Google Scholar
Digital Library
- Doe Hyun Yoon, Jichuan Chang, Naveen Muralimanohar, and Parthasarathy Ranganathan. 2012. BOOM: enabling mobile memory based low-power server DIMMs. In Proceedings of the 39th International Symposium on Computer Architecture (ISCA). IEEE Computer Society, Los Alamitos, CA, 25--36. Google Scholar
Digital Library
Index Terms
Market mechanisms for managing datacenters with heterogeneous microarchitectures
Recommendations
Breaking the Boundaries in Heterogeneous-ISA Datacenters
ASPLOS '17: Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating SystemsEnergy efficiency is one of the most important design considerations in running modern datacenters. Datacenter operating systems rely on software techniques such as execution migration to achieve energy efficiency across pools of machines. Execution ...
Managing energy, performance and cost in large scale heterogeneous datacenters using migrations
AbstractImproving datacenter energy efficiency becomes increasingly important due to energy supply problems, fuel costs and global warming. Virtualisation can help to improve datacenter energy efficiency through server consolidation which ...
Highlights- The existence of a trade-off between overall energy consumption and performance (hence cost).
Navigating heterogeneous processors with market mechanisms
HPCA '13: Proceedings of the 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA)Specialization of datacenter resources brings performance and energy improvements in response to the growing scale and diversity of cloud applications. Yet heterogeneous hardware adds complexity and volatility to latency-sensitive applications. A ...






Comments