skip to main content
research-article

Interference Management for Distributed Parallel Applications in Consolidated Clusters

Published:25 March 2016Publication History
Skip Abstract Section

Abstract

Consolidating multiple applications on a system can improve the overall resource utilization of data center systems. However, such consolidation can adversely affect the performance of some applications due to interference caused by resource contention. Despite many prior studies on the interference effects in single-node systems, the interference behaviors of distributed parallel applications have not been investigated thoroughly. With distributed applications, a local interference in a node can affect the whole execution of an application spanning many nodes. This paper studies an interference modeling methodology for distributed applications to predict their performance under interference effects in consolidated clusters. This study first characterizes the effects of interference for various distributed applications over different interference settings, and analyzes how diverse interference intensities on multiple nodes affect the overall performance. Based on the characterization, this study proposes a static profiling-based model for interference propagation and heterogeneity behaviors. In addition, this paper presents use case studies of the modeling method, two interference-aware placement techniques for consolidated virtual clusters, which attempt to maximize the overall throughput or to guarantee the quality-of-service.

References

  1. VMware ESX Server 2 NUMA Support. White paper.Google ScholarGoogle Scholar
  2. Jeongseob Ahn, Changdae Kim, Jaeung Han, Young-ri Choi, and Jaehyuk Huh. Dynamic virtual machine scheduling in clouds for architectural shared resources. In Proceedings of the 4th USENIX Conference on Hot Topics in Cloud Computing (HotCloud), 2012.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Sergey Blagodurov, Sergey Zhuravlev, Mohammad Dashti, and Alexandra Fedorova. A case for NUMA-aware contention management on multicore systems. In Proceedings of the 2011 USENIX Conference on USENIX Annual Technical Conference (ATC), 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Xiangping Bu, Jia Rao, and Cheng-zhong Xu. Interference and locality-aware task scheduling for MapReduce applications in virtual clusters. In Proceedings of the 22nd International Symposium on High-performance Parallel and Distributed Computing (HPDC), 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Ron C. Chiang and H. Howie Huang. Tracon: interference-aware scheduling for data-intensive applications in virtualized environments. In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Hyung Won Choi, Hukeun Kwak, Andrew Sohn, and Kyusik Chung. Autonomous learning for efficient resource utilization of dynamic VM migration. In Proceedings of the 22nd annual international conference on Supercomputing (ICS), 2008.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Christina Delimitrou and Christos Kozyrakis. Paragon: QoS-aware scheduling for heterogeneous datacenters. In Proceedings of the 18th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Christina Delimitrou and Christos Kozyrakis. Quasar: Resource-efficient and QoS-aware cluster management. In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2014.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. RW Eglese. Simulated annealing: a tool for operational research. European journal of operational research, 46(3):271--281, 1990.Google ScholarGoogle Scholar
  10. Ajay Gulati, Ganesha Shanmuganathan, Anne Holler, and Irfan Ahmad. Cloud-scale resource management: Challenges and techniques. In Proceedings of the 3rd USENIX Conference on Hot Topics in Cloud Computing (HotCloud), 2011.Google ScholarGoogle Scholar
  11. Jaeung Han, Jeongseob Ahn, Changdae Kim, Youngjin Kwon, Young-ri Choi, and Jaehyuk Huh. The effect of multi-core on HPC applications in virtualized systems. In Proceedings of the 5th Workshop on Virtualization in High-Performance Cloud Computing (VHPC), 2011.Google ScholarGoogle ScholarCross RefCross Ref
  12. Jason Mars and Lingjia Tang. Whare-map: Heterogeneity in "homogeneous" warehouse-scale computers. In Proceedings of the 40th Annual International Symposium on Computer Architecture (ISCA), 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jason Mars, Lingjia Tang, Robert Hundt, Kevin Skadron, and Mary Lou Soffa. Bubble-Up: Increasing utilization in modern warehouse scale computers via sensible co-locations. In Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Andreas Merkel, Jan Stoess, and Frank Bellosa. Resource-conscious scheduling for energy efficiency on multicore processors. In Proceedings of the 5th European Conference on Computer Systems (EuroSys), 2010.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Ripal Nathuji, Aman Kansal, and Alireza Ghaffarkhah. Q-clouds: managing performance interference effects for qos-aware clouds. In Proceedings of the 5th European conference on Computer systems (EuroSys), 2010.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Dejan Novaković, Nedeljko Vasić, Stanko Novaković, Dejan Kostić, and Ricardo Bianchini. DeepDive: Transparently identifying and managing performance interference in virtualized environments. In Proceedings of the 2013 USENIX Conference on Annual Technical Conference (ATC), 2013.Google ScholarGoogle Scholar
  17. Moinuddin K. Qureshi and Yale N. Patt. Utility-based cache partitioning: A low-overhead, high-performance, runtime mechanism to partition shared caches. In Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2006.Google ScholarGoogle Scholar
  18. Jason Sonnek, James Greensky, Robert Reutiman, and Abhishek Chandra. Starling: Minimizing communication overhead in virtualized computing platforms using decentralized affinity-aware migration. In Proceedings of the 2010 39th International Conference on Parallel Processing (ICPP), 2010.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. G. Edward Suh, Srinivas Devadas, and Larry Rudolph. A new memory monitoring scheme for memory-aware scheduling and partitioning. In Proceedings of the 8th International Symposium on High-Performance Computer Architecture (HPCA), 2002.Google ScholarGoogle ScholarCross RefCross Ref
  20. Timothy Wood, Prashant Shenoy, Arun Venkataramani, and Mazin Yousif. Black-box and gray-box strategies for virtual machine migration. In Proceedings of the 4th USENIX Conference on Networked Systems Design & Implementation (NSDI), 2007.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Hailong Yang, Alex Breslow, Jason Mars, and Lingjia Tang. Bubble-flux: Precise online QoS management for increased utilization in warehouse scale computers. In Proceedings of the 40th Annual International Symposium on Computer Architecture (ISCA), 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Xiao Zhang, Eric Tune, Robert Hagmann, Rohit Jnagal, Vrigo Gokhale, and John Wilkes. CPI2: Cpu performance isolation for shared compute clusters. In Proceedings of the 8th European Conference on Computer Systems (EuroSys), 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Qian Zhu, Jiedan Zhu, and Gagan Agrawal. Power-aware consolidation of scientific workflows in virtualized environments. In Proceedings of the 2010 International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2010.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Sergey Zhuravlev, Sergey Blagodurov, and Alexandra Fedorova. Addressing shared resource contention in multicore processors via scheduling. In Proceedings of the 15th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2010.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Interference Management for Distributed Parallel Applications in Consolidated Clusters

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!