Given the tremendous interest in VXLAN with MP-BGP based EVPN Control-Plane (short EVPN) at Cisco Live in Milan, I decided to write a “short” technology brief blog post on this topic.
VXLAN (IETF RFC7348) has been designed to solve specific problems faced with Classical Ethernet for a few decades now. By introducing an abstraction through encapsulation, VXLAN has become the de-facto standard overlay of choice in the industry. Chief among the advantages provided by VXLAN; extension of the todays limited VLAN space and the increase in the scalability provided for Layer-2 Domains.
Extended Namespace – The available VLAN space from the IEEE 802.1Q encapsulation perspective is limited to a 12-bit field, which provides 4096 VLANs or segments. By encapsulating the original Ethernet frame with a VXLAN header, the newly introduced addressing field offers 24-bits, thereby providing a much larger namespace with up to 16 Million Virtual Network Identifiers (VNIs) or segments.
While the VXLAN VNI allows unique identification of a large number of tenant segments which is especially useful in high-scale multi-tenant deployments, the problems and requirements of large Layer-2 Domains are not sufficiently addressed. However, significant improvements in the following areas have been achieved:
- No dependency on Spanning-Tree protocol by leveraging Layer-3 routing protocols
- Layer-3 routing with Equal Cost Multi-Path (ECMP) allows all available links to be used
- Scalability, convergence, and resiliency of a Layer-3 network
- Isolation of Broadcast and Failure Domains
IETF RFC7348 – VXLAN: A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks
Scalable Layer-2 Domains
The abstraction by using a VXLAN-like overlay does not inherently change the Flood & Learn behavior introduced by Ethernet. In typical deployments of VXLAN, BUM (Broadcast, Unicast, Multicast) traffic is forwarded via layer-3 multicast in the underlay that in turn aids in the learning process so that subsequent traffic need not be subjected to this “flood” semantic. A control-plane is required to minimize the flood behavior and proactively distribute End-Host information to participating entities (typically called Virtual Tunnel End Points aka VTEPs) in the same segment – learning.
Control-plane protocols are mostly employed in the layer-3 routing space where predominantly IP prefix information is exchanged. Over the past years, some of the well-known routing protocols have been extended to also learn and exchange Layer-2 MAC addresses. An early technology adoption with MAC addresses in a routing-protocol was Cisco’s OTV (Overlay Transport Virtualization), which employed IS-IS to significantly reduce flooding across Data Center Interconnects (DCI).
Multi-Protocol BGP (MP-BGP) introduced a new Network Layer Reachability Information (NLRI) to carry both, Layer-2 MAC and Layer-3 IP information at the same time. By having the combined set of MAC and IP information available for forwarding decisions, optimized routing and switching within a network becomes feasible and the need for flood to do learning get minimized or even eliminated. This extension that allows BGP to transport Layer-2 MAC and Layer-3 IP information is called EVPN – Ethernet Virtual Private Network.
EVPN is documented in the following IETF drafts
Integrated Route and Bridge (IRB) – VXLAN-EVPN offers significant advantages in Overlay networking by optimizing forwarding decision within the network based on Layer-2 MAC as well as Layer-3 IP information. The decision on forwarding via routing or switching can be done as close as possible to the End-Host, on any given Leaf/ToR (Top-of-Rack) Switch. The Leaf Switch provides the Distributed Anycast Gateway for routing, which acts completely stateless and does not require the exchange of protocol signalization for election or failover decision. All the reachability information available within the BGP control-plane is sufficient to provide the gateway service. The Distributed Anycast Gateway also provides integrated routing and bridging (IRB) decision at the Leaf Switch, which can be extended across a significant number of nodes. All the Leaf Switches host active default gateways for their respective configured subnets; the well known semantic of First Hop Routing Protocols (FHRP) with active/standby does not apply anymore.
Summary – The advantages provided by a VXLAN-EVPN solution are briefly summarized as follows:
- Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)
- Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)
- Forwarding decision based on Control-Plane (minimizes flooding)
- Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay
- Leverages Layer-3 ECMP – all links forwarding – in the Underlay
- Significantly larger Name-Space in the Overlay (16M segments)
- Integration of Physical and Virtual Networks with Hybrid Overlays
- It facilitates Software-Defined-Networking (SDN)
Simply formulated, VXLAN-EVPN provides a standards-based Overlay that supports Segmentation, Host Mobility, and High Scale.
VXLAN-EVPN is available on Nexus 9300 (NX-OS 7.0) with Nexus 7000/7700 (F3 linecards) to follow in the upcoming major release. Additional Data Center Switching platforms, like the Nexus 5600, will follow shortly after.
A detailed whitepaper on this topic is available on Cisco.com. In addition, VXLAN-EVPN was featured during the following Cisco Live! Sessions.
Do you have appetite for more? Post a comment, tweet about it and have the conversation going … Thanks for reading and Happy Networking!
Tags: #CLEUR, Cisco, cisco live, Cisco Nexus, Cisco Nexus 9000, data center, EVPN, ietf, network, nexus, rfc7348, SDN, VXLAN
FlexPod had a great 2014 and continues to be a leader in the integrated infrastructure market. The collaboration and execution between the Cisco and NetApp teams have delivered the following results:
- Greater than 5,000 customers across 100 countries
- More than $3B in revenue
- 80+ validated designs
- 1100+ Partners
Customer needs continue to change, Cisco and NetApp are addressing those needs by introducing new solutions, technologies, and offerings to accelerate and manage applications in the data center and at the edge. We will also enhance and validate existing solutions with our latest products and technologies. We are excited to be introducing the following new offerings, technologies, and solutions to the FlexPod portfolio:
• FlexPod with UCS Mini
• Cisco UCS B200 M4 Servers
• Cisco UCS Manager 2.2
• Cisco UCS Director
• Cisco Nexus 9000
• NetApp Data ONTAP 8.2
• FlexPod Data Center with Microsoft SharePoint 2013 and Cisco ACI
• FlexPod Datacenter with Microsoft Exchange 2013 and Cisco ACI
• FlexPod Data Center with Microsoft Private Cloud 4.0
FlexPod with UCS Mini
An increasing amount of computing is being done outside the data center at the edge. FlexPod with UCS Mini is a simple, easy to manage, and expandable solution that brings the performance and power of Cisco UCS integration in an all-in-one, small footprint optimized for non-datacenter environments, such as remote sites, branch offices, and any location where data is generated and compute resources are needed. The UCS Management portfolio enables remote operation, automation and policy enforcement across massive multi-site footprints, enabling customers who have invested in FlexPod in their core data centers to leverage that investment at their smaller remote and branch offices.
We are looking forward to an exciting 2015 with FlexPod. These new solutions and technologies will enable FlexPod to meet a wide range of IT needs and run the most business-critical applications whether they are located in the data center or at the edge. To learn more about our FlexPod portfolio visit our website and to see all our validated designs check out the DesignZone for FlexPod.
If you are involved in designing, supporting or managing a data center, you will undoubtedly rely on technical support services from one or more vendors. Running your data center, there is always the risk of a hardware failure or being impacted by a software defect. While relatively rare, hardware does occasionally fail unfortunately. However you undoubtedly have technical support in place to deal with such problems. You may have invested in a few extra switches as backup, you may also have failover mechanisms in place. Almost certainly you will have a support contract in place with your Cisco partner or with Cisco, so you have break/fix expertise on tap for when something goes wrong. This is critical support for your business, no debate from me.
Engineer Under Stress!
Now, arguably the most important resource you have in your data center is not so much individual switches, routers or servers. It’s your engineers, those who design and support your data center. If they have a problem, where and how do they get help? Who helps them when they are stretched? When business pressures are telling? Of course, their colleagues and managers can and will help. Where, however, can they tap into additional sources of expertise so that they can become even more productive for you? This is where Cisco Optimization Services come in – including our award-winning Cisco Network Optimization Service (or “NOS” for short), Collaboration Optimization Service, and the one I’m involved with, Cisco Data Center Optimization Services.
Read More »
Tags: ACI, architecture, Cisco Nexus, Cisco UCS, cisco_services, data_center, OpenStack, optimization, SDN
At Cisco Live Milan, Jim McHugh, Vice President, Cisco UCS and Data Center Solutions Marketing and John Lockyer, Chief Technology Officer at VCE, give an overview of Vblock Systems management featuring VCE Vision and Cisco UCS Director. Watch the video and learn how managing converged infrastructure holistically can accelerate innovation through automation and orchestration resulting in faster ROI.
Software Defined Networking (SDN) is an exciting evolution in IT. Today, connections throughout a network are made by directly connecting to routers to control them. This is a process that can be time-consuming and prone to error, depending upon the size of the network being managed. Furthermore, the effort involved in making changes across large networks can slow the pace of innovation within an organization.
SDN moves control to the application layer. This provides numerous benefits to IT:
- Unified point of automation and management
- Consistent deployment of policies throughout the network
- Improved operational efficiency
- Rapid deployment of new features like security and QoS
- Simplified network management overall
Many people think SDN is the future of data center networking. Cloud provider OneNeck, for example, is investing in developing the expertise to lead the market in transitioning its customers to SDN. In their blog, Leading the Market with Software Defined Networking (SDN), OneNeck describes how they are working with Cisco to bring SDN to market.
OneNeck’s SDN lab, for example, is a great example of how our partners add value to Cisco technology. SDN introduces a whole new management framework for networking. By working with real-world SDN applications in their lab, OneNeck is developing the skill sets and expertise required to enables its customers to evolve with confidence.
Learn more about Cisco’s SDN solutions, including the Cisco Nexus 9000 series switches, Application Centric Infrastructure (ACI) and Cisco Intercloud Fabric.
Tags: Cisco Powered, OneNeck, SDN