WCMP: weighted cost multipathing for improved fairness in data centers

ABSTRACT

Data Center topologies employ multiple paths among servers to deliver scalable, cost-effective network capacity. The simplest and the most widely deployed approach for load balancing among these paths, Equal Cost Multipath (ECMP), hashes flows among the shortest paths toward a destination. ECMP leverages uniform hashing of balanced flow sizes to achieve fairness and good load balancing in data centers. However, we show that ECMP further assumes a balanced, regular, and fault-free topology, which are invalid assumptions in practice that can lead to substantial performance degradation and, worse, variation in flow bandwidths even for same size flows.

We present a set of simple algorithms that achieve Weighted Cost Multipath (WCMP) to balance traffic in the data center based on the changing network topology. The state required for WCMP is already disseminated as part of standard routing protocols and it can be readily implemented in the current switch silicon without any hardware modifications. We show how to deploy WCMP in a production OpenFlow network environment and present experimental and simulation results to show that variation in flow bandwidths can be reduced by as much as 25X by employing WCMP relative to ECMP.

References

1. J. H. Ahn, N. Binkert, A. Davis, M. McLaren, and R. S. Schreiber. HyperX: Topology, Routing, and Packaging of Efficient Large-Scale Networks. In Proc. of SC, November 2009.
2. M. Al-Fares, A. Loukissas, and A. Vahdat. A Scalable, Commodity Data Center Network Architecture. In Proc. of ACM SIGCOMM, August 2008.
3. M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, and A. Vahdat. Hedera: Dynamic Flow Scheduling for Data Center Networks. In Proc. of Usenix NSDI, April 2010.
4. D. Applegate, L. Breslau, and E. Cohen. Coping with Network Failures: Routing Strategies for Optimal Demand Oblivious Restoration. In Proc. of SIGMETRICS, June 2004.
5. L. A. Barroso, J. Dean, and U. Hölzle. Web Search for a Planet: The Google Cluster Architecture. IEEE Micro, Volume 23, March-April 2003.
6. Broadcom's OpenFlow Data Plane Abstraction. http://www.broadcom.com/products/Switching/Software-Defined-Networking-Solutions/OF-DPA-Software.
7. T. Benson, A. Akella, and D. A. Maltz. Network Traffic Characteristics of Data Centers in the Wild. In Proc. of ACM IMC, November 2010.
8. M. Chowdhury, M. Zaharia, J. Ma, M. I. Jordan, and I. Stoica. Managing Data Transfers in Computer Clusters with Orchestra. In Proc. of ACM SIGCOMM, August 2011.
9. T. H. Cormen, C. Stein, R. L. Rivest, and C. E. Leiserson. Introduction to Algorithms. McGraw-Hill Higher Education, 2nd edition, 2001.
10. A. R. Curtis, W. Kim, and P. Yalagandula. Mahout: Low-Overhead Datacenter Traffic Management using End-Host-Based Elephant Detection. In Proc. of IEEE INFOCOM, April 2011.
11. A. R. Curtis, J. C. Mogul, J. Tourrilhes, P. Yalagandula, P. Sharma, and S. Banerjee. DevoFlow: Scaling Flow Management For High-Performance Networks. In Proc. of ACM SIGCOMM, August 2011.
12. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In Proc. of Usenix OSDI, December 2004.
13. Gnu Linear Programming Kit. http://www.gnu.org/software/glpk.
14. A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. VL2: A Scalable and Flexible Data Center Network. In Proc. of ACM SIGCOMM, August 2009.
15. C. Guo, G. Lu, D. Li, H. Wu, X. Zhang, Y. Shi, C. Tian, Y. Zhang, and S. Lu. BCube: A High Performance, Server-centric Network Architecture for Modular Data Centers. In Proc. of ACM SIGCOMM, August 2009.
16. B. Heller, S. Seetharaman, P. Mahadevan, Y. Yiakoumis, P. Sharma, S. Banerjee, and N. McKeown. ElasticTree: Saving Energy In Data Center Networks. In Proc. of Usenix NSDI, April 2010.
17. C. Hopps. Analysis of an Equal-Cost Multi-Path Algorithm. RFC 2992, IETF, November 2000.
18. HP SDN Ecosystem. http://h17007.www1.hp.com/us/en/networking/solutions/technology/sdn/.
19. MPTCP htsim implementation. http://nrg.cs.ucl.ac.uk/mptcp/implementation.html.
20. J. Kim and W. J. Dally. Flattened Butterfly: A Cost-Efficient Topology for High-Radix Networks. In ISCA, June 2007.
21. M. Kodialam, T. V. Lakshman, and S. Sengupta. Efficient and Robust Routing of Highly Variable Traffic. In Proc. of ACM HotNets, November 2004.
22. T. Koponen, M. Casado, N. Gude, J. Stribling, L. Poutievski, M. Zhu, R. Ramanathan, Y. Iwata, H. Inoue, T. Hama, and S. Shenker. Onix: A Distributed Control Platform for Large-scale Production Networks. In Proc. of Usenix OSDI, October 2010.
23. V. Liu, D. Halperin, A. Krishnamurhty, and T. Anderson. F10: A Fault-Tolerant Engineered Network. In Proc. of Usenix NSDI, April 2013.
24. Memcached. http://www.memcached.org.
25. J. Moy. OSPF Version 2. RFC 2328, Internet Engineering Task Force, April 1998.
26. MPLS-TE. http://blog.ioshints.info/2007/02/unequal-cost-load-sharing.html.
27. Openflow. www.openflow.org.
28. C. Raiciu, S. Barre, C. Pluntke, A. Greenhalgh, D. Wischik, and M. Handley. Improving Datacenter Performance and Robustness with Multipath TCP. In Proc. of ACM SIGCOMM, August 2011.
29. C. Villamizar. OSPF Optimized Multipath (OSPF-OMP), draft-ietf-ospf-omp-02, February 1999.
30. H. Wu, G. Lu, D. Li, C. Guo, and Y. Zhang. MDCube: A High Performance Network Structure for Modular Data Center Interconnection. In Proc. of ACM CoNEXT, December 2009.
31. X. Xiao, A. Hannan, and B. Bailey. Traffic Engineering with MPLS in the Internet. IEEE Network Magazine, 2000.
32. Y. Zhang and Z. Ge. Finding Critical Traffic Matrices. In Proc. of DSN, June 2005.

Index Terms

• Networks

  • Network protocols