research-article

VL2: a scalable and flexible data center network

Authors:

Albert Greenberg,

James R. Hamilton,

Srikanth Kandula,

Parantap Lahiri,

David A. Maltz,

Parveen Patel, and

Sudipta SenguptaAuthors Info & Claims

ACM SIGCOMM Computer Communication Review, Volume 39, Issue 4

Pages 51 - 62

https://doi.org/10.1145/1594977.1592576

Published: 16 August 2009 Publication History

Abstract

To be agile and cost effective, data centers should allow dynamic resource allocation across large server pools. In particular, the data center network should enable any server to be assigned to any service. To meet these goals, we present VL2, a practical network architecture that scales to support huge data centers with uniform high capacity between servers, performance isolation between services, and Ethernet layer-2 semantics. VL2 uses (1) flat addressing to allow service instances to be placed anywhere in the network, (2) Valiant Load Balancing to spread traffic uniformly across network paths, and (3) end-system based address resolution to scale to large server pools, without introducing complexity to the network control plane. VL2's design is driven by detailed measurements of traffic and fault data from a large operational cloud service provider. VL2's implementation leverages proven network technologies, already available at low cost in high-speed hardware implementations, to build a scalable and reliable network architecture. As a result, VL2 networks can be deployed today, and we have built a working prototype. We evaluate the merits of the VL2 design using measurement, analysis, and experiments. Our VL2 prototype shuffles 2.7 TB of data among 75 servers in 395 seconds - sustaining a rate that is 94% of the maximum possible.

References

[1]

M. Al-Fares, A. Loukissas, and A. Vahdat. A scalable, commodity data center network architecture. In SIGCOMM, 2008.

Digital Library

[2]

C. Chang, D. Lee, and Y. Jou. Load balanced Birkhoff-von Neumann switches, part I: one-stage buffering. IEEE HPSR, 2001.

[3]

Cisco. Data center Ethernet. http://www.cisco.com/go/dce.

[4]

Cisco: Data center: Load balancing data center services, 2004.

[5]

K. C. Claffy, H. werner Braun, and G. C. Polyzos. A parameterizable methodology for Internet traffic flow profiling. JSAC, 13, 1995.

Digital Library

[6]

W. J. Dally and B. Towles. Principles and Practices of Interconnection Networks. Morgan Kaufmann Publishers, 2004.

Digital Library

[7]

N. G. Duffield, P. Goyal, A. G. Greenberg, P. P. Mishra, K. K. Ramakrishnan, and J. E. van der Merwe. A flexible model for resource management in virtual private network. In SIGCOMM, 1999.

Digital Library

[8]

D. Farinacci, V. Fuller, D. Oran, D. Meyer, and S. Brim. Locator/ID Separation Protocol (LISP). Internet-draft, Dec. 2008.

[9]

A. Greenberg, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. Towards a next generation data center architecture: Scalability and commoditization. In PRESTO Workshop at SIGCOMM, 2008.

Digital Library

[10]

C. Guo, H. Wu, K. Tan, L. Shiy, Y. Zhang, and S. Lu. Dcell: A scalable and fault-tolerant network structure for data centers. In SIGCOMM, 2008.

Digital Library

[11]

C. Guo, H. Wu, K. Tan, L. Shiy, Y. Zhang, and S. Lu. Bcube: A high performance, server-centric network architecture for modular data centers. In SIGCOMM, 2009.

Digital Library

[12]

J. Hamilton. Cems: Low-cost, low-power servers for internet-scale services. In Conf. on Innovative Data Systems Research, Jan 2009.

[13]

M. Handley, S. Floyd, J. Padhye, and J. Widmer. TCP friendly rate control (TFRC): Protocol specification. RFC 3348, 2003.

Digital Library

[14]

R. Jain. The Art of Computer Systems Performance Analysis. John Wiley and Sons, Inc., 1991.

[15]

S. Kandula, D. Katabi, B. Davie, and A. Charny. Walking the Tightrope: Responsive yet Stable Traffic Engineering. In SIGCOMM, 2005.

Digital Library

[16]

C. Kim, M. Caesar, and J. Rexford. Floodless in SEATTLE: a scalable ethernet architecture for large enterprises. In SIGCOMM, 2008.

Digital Library

[17]

M. Kodialam, T. V. Lakshman, and S. Sengupta. Efficient and Robust Routing of Highly Variable Traffic. In HotNets, 2004.

[18]

L. Lamport. The part-time parliament. ACM Transactions on Computer Systems, 16:133--169, 1998.

Digital Library

[19]

M. Mathis, J. Heffner, and R. Raghunarayan. TCP extended statistics MIB. RFC 4898, 2007.

[20]

S. Sinha, S. Kandula, and D. Katabi. Harnessing TCP's burstiness with flowlet switching. In HotNets, 2004.

[21]

Y. Zhang and Z. Ge. Finding critical traffic matrices. In DSN, June 2005.

[22]

R. Zhang-Shen and N. McKeown. Designing a Predictable Internet Backbone Network. In HotNets, 2004.

Cited By

Gao WZhong JPeng CLi XLiao X(2024)Fine-grained load balancing with proactive prediction and adaptive rerouting in data centerJournal of High Speed Networks10.3233/JHS-23000330:1(83-96)Online publication date: 10-Jan-2024
https://doi.org/10.3233/JHS-230003
Ferreira MAtre NSherry JSobrinho JKuznetsov PGelles ROlivetti D(2024)Impossibility Results for Data-Center Routing with Congestion Control and Unsplittable FlowsProceedings of the 43rd ACM Symposium on Principles of Distributed Computing10.1145/3662158.3662777(358-368)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3662158.3662777
Burke QMcDaniel PLa Porta TYu MHe T(2024)Misreporting Attacks Against Load Balancers in Software-Defined NetworkingMobile Networks and Applications10.1007/s11036-023-02156-0Online publication date: 9-Jan-2024
https://doi.org/10.1007/s11036-023-02156-0
Show More Cited By

Index Terms

VL2: a scalable and flexible data center network
1. Networks
  1. Network protocols

Recommendations

VL2: a scalable and flexible data center network
SIGCOMM '09: Proceedings of the ACM SIGCOMM 2009 conference on Data communication

To be agile and cost effective, data centers should allow dynamic resource allocation across large server pools. In particular, the data center network should enable any server to be assigned to any service. To meet these goals, we present VL2, a ...
Read More
Data center TCP (DCTCP)
SIGCOMM '10

Cloud data centers host diverse applications, mixing workloads that require small predictable latency with others requiring large sustained throughput. In this environment, today's state-of-the-art TCP protocol falls short. We present measurements of a ...
Read More
Data center TCP (DCTCP)
SIGCOMM '10: Proceedings of the ACM SIGCOMM 2010 conference

Cloud data centers host diverse applications, mixing workloads that require small predictable latency with others requiring large sustained throughput. In this environment, today's state-of-the-art TCP protocol falls short. We present measurements of a ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM SIGCOMM Computer Communication Review

ACM SIGCOMM Computer Communication Review Volume 39, Issue 4

SIGCOMM '09

October 2009

325 pages

ISSN:0146-4833

DOI:10.1145/1594977

Issue’s Table of Contents

SIGCOMM '09: Proceedings of the ACM SIGCOMM 2009 conference on Data communication
August 2009
340 pages
ISBN:9781605585949
DOI:10.1145/1592568
General Chairs:
Pablo Rodriguez
Telefonica Research, Spain
,
Ernst Biersack
Eurecom, France
,
Program Chairs:
Konstantina Papagiannaki
Intel Labs Pittsburgh, USA
,
Luigi Rizzo
Università di Pisa, Italy

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 August 2009

Published in SIGCOMM-CCR Volume 39, Issue 4

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2,000
Total Citations
View Citations
11,536
Total Downloads

Downloads (Last 12 months)1,055
Downloads (Last 6 weeks)88

Other Metrics

View Author Metrics

Citations

Cited By

Gao WZhong JPeng CLi XLiao X(2024)Fine-grained load balancing with proactive prediction and adaptive rerouting in data centerJournal of High Speed Networks10.3233/JHS-23000330:1(83-96)Online publication date: 10-Jan-2024
https://doi.org/10.3233/JHS-230003
Ferreira MAtre NSherry JSobrinho JKuznetsov PGelles ROlivetti D(2024)Impossibility Results for Data-Center Routing with Congestion Control and Unsplittable FlowsProceedings of the 43rd ACM Symposium on Principles of Distributed Computing10.1145/3662158.3662777(358-368)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3662158.3662777
Burke QMcDaniel PLa Porta TYu MHe T(2024)Misreporting Attacks Against Load Balancers in Software-Defined NetworkingMobile Networks and Applications10.1007/s11036-023-02156-0Online publication date: 9-Jan-2024
https://doi.org/10.1007/s11036-023-02156-0
Vatsal SVerma D(2023)Data Centre Efficiency Enhancement by Metrics Oriented Approach to Revamp Green Cloud Computing ConceptInternational Journal of Innovative Technology and Exploring Engineering10.35940/ijitee.F9532.071282312:8(1-14)Online publication date: 30-Jul-2023
https://doi.org/10.35940/ijitee.F9532.0712823
Zadeh SZandi FBuckley MGanjali Y(2023)Meta-Migration: Reducing Switch Migration Tail Latency Through Competition2023 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking57963.2023.10186446(1-9)Online publication date: 12-Jun-2023
https://doi.org/10.23919/IFIPNetworking57963.2023.10186446
Gao WHuang JLi ZZou SWang J(2023)PDLB: Path Diversity-aware Load Balancing with adaptive granularity in data center networksJournal of Cloud Computing10.1186/s13677-023-00548-x12:1Online publication date: 7-Dec-2023
https://doi.org/10.1186/s13677-023-00548-x
Liu HUrata RYasumura KZhou XBannon RBerger JDashti PJouppi NLam CLi SMao ENelson DPapen GTariq MVahdat ASchulzrinne HKohler EMaltz DMisra V(2023)Lightwave Fabrics: At-Scale Optical Circuit Switching for Datacenter and Machine Learning SystemsProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604836(499-515)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3603269.3604836
Gozlan IAvin CEinziger GScalosub G(2023)Go-to-Controller is Better: Efficient and Optimal LPM Caching with SplicingProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/35794417:1(1-33)Online publication date: 2-Mar-2023
https://doi.org/10.1145/3579441
Liu HLi WPang YPei RHu YLiu YSuo LLi K(2023)Accelerating Data Delivery of Latency-Sensitive Applications in Container Overlay NetworkIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.330074534:12(3046-3058)Online publication date: Dec-2023
https://doi.org/10.1109/TPDS.2023.3300745
Zhao JLiu JWang HXu CZhang H(2023)Multipath Congestion Control: Measurement, Analysis, and Optimization From the Energy PerspectiveIEEE Transactions on Network Science and Engineering10.1109/TNSE.2023.3257034(1-12)Online publication date: 2023
https://doi.org/10.1109/TNSE.2023.3257034
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents