Cisco Blog > Data Center and Cloud
Given the tremendous interest in VXLAN with MP-BGP based EVPN Control-Plane (short EVPN) at Cisco Live in Milan, I decided to write a “short” technology brief blog post on this topic.
VXLAN (IETF RFC7348) has been designed to solve specific problems faced with Classical Ethernet for a few decades now. By introducing an abstraction through encapsulation, VXLAN has become the de-facto standard overlay of choice in the industry. Chief among the advantages provided by VXLAN; extension of the todays limited VLAN space and the increase in the scalability provided for Layer-2 Domains.
Extended Namespace – The available VLAN space from the IEEE 802.1Q encapsulation perspective is limited to a 12-bit field, which provides 4096 VLANs or segments. By encapsulating the original Ethernet frame with a VXLAN header, the newly introduced addressing field offers 24-bits, thereby providing a much larger namespace with up to 16 Million Virtual Network Identifiers (VNIs) or segments.
While the VXLAN VNI allows unique identification of a large number of tenant segments which is especially useful in high-scale multi-tenant deployments, the problems and requirements of large Layer-2 Domains are not sufficiently addressed. However, significant improvements in the following areas have been achieved:
- No dependency on Spanning-Tree protocol by leveraging Layer-3 routing protocols
- Layer-3 routing with Equal Cost Multi-Path (ECMP) allows all available links to be used
- Scalability, convergence, and resiliency of a Layer-3 network
- Isolation of Broadcast and Failure Domains
IETF RFC7348 – VXLAN: A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks
Scalable Layer-2 Domains
The abstraction by using a VXLAN-like overlay does not inherently change the Flood & Learn behavior introduced by Ethernet. In typical deployments of VXLAN, BUM (Broadcast, Unicast, Multicast) traffic is forwarded via layer-3 multicast in the underlay that in turn aids in the learning process so that subsequent traffic need not be subjected to this “flood” semantic. A control-plane is required to minimize the flood behavior and proactively distribute End-Host information to participating entities (typically called Virtual Tunnel End Points aka VTEPs) in the same segment – learning.
Control-plane protocols are mostly employed in the layer-3 routing space where predominantly IP prefix information is exchanged. Over the past years, some of the well-known routing protocols have been extended to also learn and exchange Layer-2 MAC addresses. An early technology adoption with MAC addresses in a routing-protocol was Cisco’s OTV (Overlay Transport Virtualization), which employed IS-IS to significantly reduce flooding across Data Center Interconnects (DCI).
Multi-Protocol BGP (MP-BGP) introduced a new Network Layer Reachability Information (NLRI) to carry both, Layer-2 MAC and Layer-3 IP information at the same time. By having the combined set of MAC and IP information available for forwarding decisions, optimized routing and switching within a network becomes feasible and the need for flood to do learning get minimized or even eliminated. This extension that allows BGP to transport Layer-2 MAC and Layer-3 IP information is called EVPN – Ethernet Virtual Private Network.
EVPN is documented in the following IETF drafts
Integrated Route and Bridge (IRB) – VXLAN-EVPN offers significant advantages in Overlay networking by optimizing forwarding decision within the network based on Layer-2 MAC as well as Layer-3 IP information. The decision on forwarding via routing or switching can be done as close as possible to the End-Host, on any given Leaf/ToR (Top-of-Rack) Switch. The Leaf Switch provides the Distributed Anycast Gateway for routing, which acts completely stateless and does not require the exchange of protocol signalization for election or failover decision. All the reachability information available within the BGP control-plane is sufficient to provide the gateway service. The Distributed Anycast Gateway also provides integrated routing and bridging (IRB) decision at the Leaf Switch, which can be extended across a significant number of nodes. All the Leaf Switches host active default gateways for their respective configured subnets; the well known semantic of First Hop Routing Protocols (FHRP) with active/standby does not apply anymore.
Summary – The advantages provided by a VXLAN-EVPN solution are briefly summarized as follows:
- Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)
- Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)
- Forwarding decision based on Control-Plane (minimizes flooding)
- Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay
- Leverages Layer-3 ECMP – all links forwarding – in the Underlay
- Significantly larger Name-Space in the Overlay (16M segments)
- Integration of Physical and Virtual Networks with Hybrid Overlays
- It facilitates Software-Defined-Networking (SDN)
Simply formulated, VXLAN-EVPN provides a standards-based Overlay that supports Segmentation, Host Mobility, and High Scale.
VXLAN-EVPN is available on Nexus 9300 (NX-OS 7.0) with Nexus 7000/7700 (F3 linecards) to follow in the upcoming major release. Additional Data Center Switching platforms, like the Nexus 5600, will follow shortly after.
A detailed whitepaper on this topic is available on Cisco.com. In addition, VXLAN-EVPN was featured during the following Cisco Live! Sessions.
Do you have appetite for more? Post a comment, tweet about it and have the conversation going … Thanks for reading and Happy Networking!
Tags: #CLEUR, Cisco, cisco live, Cisco Nexus, Cisco Nexus 9000, data center, EVPN, ietf, network, nexus, rfc7348, SDN, VXLAN
In the past, we have pointed out that configuring network services and security policies into an application network has traditionally been the most complex, tedious and time-consuming aspect of deploying new applications. For a data center or cloud provider to stand up applications in minutes and not days, easily configuring the right service nodes (e.g. a load balancer or firewall), with the right application and security policies, to support the specific workload requirements, independent of location in the network is a clear obstacle that has to be overcome.
Let’s say, for example, you have a world-beating best-in-class firewall positioned in some rack of your data center. You also have two workloads that need to be separated according to security policies implemented on this firewall on other servers a few hops away. The network and security teams have traditionally had a few challenges to address:
- If traffic from workload1 to workload2 needs to go through a firewall, how do you route traffic properly, considering the workloads don’t themselves have visibility to the specifics of the firewalls they need to work with. Traffic routing of this nature can be implemented in the network through the use of VLAN’s and policy-based routing techniques, but this is not scalable to hundreds or thousands of applications, is tedious to manage, limits workload mobility, and makes the whole infrastructure more error-prone and brittle.
- The physical location of the firewall or network service largely determines the topology of the network, and have historically restricted where workloads could be placed. But modern data center and cloud networks need to be able to provide required services and policies independent of where the workloads are placed, on this rack or that, on-premises or in the cloud.
Whereas physical firewalls might have been incorporated into an application network through VLAN stitching, there are a number of other protocols and techniques that generally have to be used with other network services to include them in an application deployment, such as Source NAT for application delivery controllers, or WCCP for WAN optimization. The complexity of configuring services for a single application deployment thus increases measurably.
Read More »
Tags: ACI, ietf, Network Services Header, Nexus 1000v, NSH, SDN, vPath
As the IETF (Internet Engineering Task Force) meets in Hawaii (IETF 91), the unavoidable question for both participants and observers is whether a Standards Development Organization (SDO) like the IETF is relevant in a rapidly expanding environment of Open Source Software (OSS) projects.
For those new to the conversation, the open question is NOT whether SDOs should exist. They are a political reality inexorably tied to trade policies and international relationships. The fundamental reason behind their existence is to avoid a communications Tower of Babel (with the resulting economic consequences) and establish governance over the use of global commercial and information infrastructure (not just acceptable behavior, but the management of resources like addressing as well). Rather, the question is about their role going forward in enabling innovation.
SDO Challenges
SDOs (like the IETF) have to evolve their processes Read More »
Tags: API, carrier ethernet, ietf, network function virtualization, Open Loop, open source, Open Stack, open standards, OSS, SDN NFV, SDO, service providers, software defined network, yang
Voice and video communications over IP have become ubiquitous over the last decade, pervasive across desktop apps, mobile apps, IP phones, video conferencing endpoints, and more. One big barrier remains: users can’t collaborate directly from their web browser without downloading cumbersome plugins for different applications. WebRTC – a set of extensions to HTML5 – can change that and enable collaboration from any browser. However, one of the major stumbling blocks in adoption of this technology is a common codec for real-time video.
The Internet Engineering Task Force (IETF) and World Wide Web Consortium (W3C) have been working jointly to standardize on the right video codec for WebRTC. Cisco and many others have been strong proponents of the H.264 industry standard codec. In support of this, almost a year ago Cisco announced that we would be open sourcing our H.264 codec and providing the source code, as well as a binary module that can be downloaded for free from the Internet. Perhaps most importantly, we announced that we would not pass on our MPEG-LA licensing costs for this binary module, making it effectively free for applications to download the module and communicate with the millions of other H.264 devices. At that time, Mozilla announced its plans to add H.264 support to Firefox using OpenH264.
Since then, we’ve made enormous progress in delivering on that promise. We open sourced the code, set up a community and website to maintain it, delivered improvements and fixes, published the binary module, and have made it available to all. This code has attracted a community of developers that helped improve Read More »
Tags: ericsson, firefox, H.264, html5, ietf, Mozilla, open source, OpenH264, video, W3C, WebRTC
Continuing on its tradition of contributing and committing to open source and open standards over the last 25 years, today Cisco announced “OpFlex” – a new open standards-based protocol for Application Centric Infrastructure that has been submitted into the IETF standardization process. We believe this will accelerate multi-vendor innovation in data center and cloud networks to drive operational simplicity, lower costs and increased agility.
Why is this required?
Traditional SDN models today function on the basis of an imperative control model with a centralized controller and distributed network entities that support the lowest common denominator feature set across vendors such as bridges, ports and tunnels. As the network scales, the controller becomes a bottleneck due to the need to maintain increased state, and starts to impact performance and resiliency. Likewise, because the applications, ops and infrastructure requirements need to be translated into network configuration, it impacts agility and introduces a manual learning process, requiring app developers to describe their requirements in low-level constructs.
If we contrast that with the vision of the ACI model with the Application Policy Infrastructure Controller (APIC), ACI adopts a declarative management approach. This model abstracts applications, operations and infrastructure providing simplification and agility. By distributing complexity to the edges, it also increases better scalability, and allows for resiliency – i.e. the data forwarding can still continue to happen even if there is no controller. It further provides ease of use with self-documenting policies automatically deployed or cleaned up from devices as necessary. All of these help circumvent the issues seen in traditional SDN models.
For this declarative model to work across a multi-vendor environment, to translate and map policy definition into the infrastructure, there has hitherto been no standard protocol to do that across physical/virtual switches, routers and L4-L7 network services. This vacuum has led to the development of “OpFlex” – a new open standard recently submitted to the IETF.
Who is contributing to OpFlex?
Several industry leaders and practitioners are actively involved in the standardization process. These include Microsoft, IBM, Citrix and SunGard Availability Services, in addition to Cisco.
Read More »
Tags: APIC, application centric infrastructure, AVI networks, Canonical, citrix, Embrane, F5, IBM, ietf, Microsoft, OpFlex, Shashi Kiran, SunGard Availability Services
