Cisco Blogs

Cisco Blog > Perspectives

A Summary of Cisco VXLAN Control Planes: Multicast, Unicast, MP-BGP EVPN

With the adoption of overlay networks as the standard deployment for multi-tenant network, Layer2 over Layer3 protocols have been the favorite among network engineers. One of the Layer2 over Layer3 (or Layer2 over UDP) protocols adopted by the industry is VXLAN. Now, as with any other overlay network protocol, its scalability is tied into how well it can handle the Broadcast, Unknown unicast and Multicast (BUM). That is where the evolution of VXLAN control plane comes into play.

The standard does not define a “standard” control plane for VXLAN. There are several drafts describing the use of different control planes. The most commonly use VXLAN control plane is multicast. It is implemented and supported by multiple vendors and it is even natively supported in server OS like the Linux Kernel.

This post tries to summarize the three (3) control planes currently supported by some of the Cisco NX-OS/IOS-XR. My focus is more towards the Nexus 7k, Nexus 9k, Nexus 1k and CSR1000v.

Each control plane may have a series of caveats in their own, but those are not covered by this blog entry. Let’s start with some VXLAN definitions:

(1) VXLAN Tunnel Endpoint (VTEP): Map tenants’ end devices to VXLAN segments. Used to perform VXLAN encapsulation/de-encapsulation.
(2) Virtual Network Identifier (VNI): identify a VXLAN segment. It hast up to 224 IDs theoretically giving us 16,777,216 segments. (Valid VNI values are from 4096 to 16777215). Each segment can transport 802.1q-encapsulated packets, theoretically giving us 212 or 4096 VLANs over a single VNI.
(3) Network Virtualization Endpoint or Network Virtualization Edge (NVE): overlay interface configured in Cisco devices to define a VTEP

VXLAN with Multicast Control Plane

Read More »

Tags: , , , , , ,

#CiscoChampion Radio S2|Ep 8. ACI with @Malhoit

CiscoChampion2015200PX#CiscoChampion Radio is a podcast series by Cisco Champions as technologists. Today we’ll be talking about ACI with Cisco Technology Evangelist Lauren Malhoit and Cisco Technical Marketing Engineer Carly Stoughton.

Listen to the Podcast.

Learn about the Cisco Champions Program HERE.
See a list of all #CiscoChampion Radio podcasts HERE.

Cisco SMEs
Lauren Malhoit, @malhoit, Cisco Technology Evangelist
Carly Stoughton, _vCarly, Cisco Technical Marketing Engineer

Cisco Champion Guest Hosts
Michael Aossey, @aossey, Solutions Architect
Chris Nickl, @ck_nic, Cloud Infrastructure Architect Read More »

Tags: , , ,

No Latency Load Balancer in Data Centers

Cisco Intelligent Traffic Director (ITD) is a zero latency multi-terabit layer 4 load-balancer available on 5k/6k/7k/9k. It has support for traffic steering and clustering solution on the Nexus  series of switches.

ITD allows customers to deploy servers and appliances from any vendor with no network or topology changes. With a few simple configuration steps on a Cisco Nexus  switches, customers can create an appliance or server cluster and deploy multiple devices to scale service capacity with ease. The servers or appliances do not have to be directly connected to the Cisco Nexus switch. ITD takes load balancing decision in a single clock cycle of the switching hardware, Hence avoiding latency altogether.

ITD supports IP-stickiness, resiliency, NAT (EFT), VIP, health monitoring, sophisticated failure handling policies, N+M redundancy, IPv4, IPv6, VRF, weighted load-balancing, bi-directional flow-coherency, and IPSLA probes including DNS. There is no service module or external appliance needed.

ITD provides order of magnitude CAPEX and OPEX savings for the customers. ITD is available on Nexus 7000/7700 series in NX-OS 6.2(8) or later. It is available for demo/EFT on Nexus 5k/6k/9k.  ITD is much superior than legacy solutions like PBR, WCCP, ECMP, port-channel, layer-4 load-balancer appliances.

Screen Shot 2015-03-02 at 10.28.21 AM

Note:- ITD is not a replacement for  L7 load balancers



Tags: , , , , , , ,

VXLAN/EVPN: Standards based Overlay with Control-Plane

Given the tremendous interest in VXLAN with MP-BGP based EVPN Control-Plane (short EVPN) at Cisco Live in Milan, I decided to write a “short” technology brief blog post on this topic.

VXLAN (IETF RFC7348) has been designed to solve specific problems faced with Classical Ethernet for a few decades now. By introducing an abstraction through encapsulation, VXLAN has become the de-facto standard overlay of choice in the industry. Chief among the advantages provided by VXLAN; extension of the todays limited VLAN space and the increase in the scalability provided for Layer-2 Domains.

Extended Namespace – The available VLAN space from the IEEE 802.1Q encapsulation perspective is limited to a 12-bit field, which provides 4096 VLANs or segments. By encapsulating the original Ethernet frame with a VXLAN header, the newly introduced addressing field offers 24-bits, thereby providing a much larger namespace with up to 16 Million Virtual Network Identifiers (VNIs) or segments.




While the VXLAN VNI allows unique identification of a large number of tenant segments which is especially useful in high-scale multi-tenant deployments, the problems and requirements of large Layer-2 Domains are not sufficiently addressed. However, significant improvements in the following areas have been achieved:

  • No dependency on Spanning-Tree protocol by leveraging Layer-3 routing protocols
  • Layer-3 routing with Equal Cost Multi-Path (ECMP) allows all available links to be used
  • Scalability, convergence, and resiliency of a Layer-3 network
  • Isolation of Broadcast and Failure Domains

IETF RFC7348 – VXLAN: A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks

Scalable Layer-2 Domains

The abstraction by using a VXLAN-like overlay does not inherently change the Flood & Learn behavior introduced by Ethernet. In typical deployments of VXLAN, BUM (Broadcast, Unicast, Multicast) traffic is forwarded via layer-3 multicast in the underlay that in turn aids in the learning process so that subsequent traffic need not be subjected to this “flood” semantic. A control-plane is required to minimize the flood behavior and proactively distribute End-Host information to participating entities (typically called Virtual Tunnel End Points aka VTEPs) in the same segment – learning.

Control-plane protocols are mostly employed in the layer-3 routing space where predominantly IP prefix information is exchanged. Over the past years, some of the well-known routing protocols have been extended to also learn and exchange Layer-2 MAC addresses. An early technology adoption with MAC addresses in a routing-protocol was Cisco’s OTV (Overlay Transport Virtualization), which employed IS-IS to significantly reduce flooding across Data Center Interconnects (DCI).

Multi-Protocol BGP (MP-BGP) introduced a new Network Layer Reachability Information (NLRI) to carry both, Layer-2 MAC and Layer-3 IP information at the same time. By having the combined set of MAC and IP information available for forwarding decisions, optimized routing and switching within a network becomes feasible and the need for flood to do learning get minimized or even eliminated. This extension that allows BGP to transport Layer-2 MAC and Layer-3 IP information is called EVPN – Ethernet Virtual Private Network.

EVPN is documented in the following IETF drafts

Integrated Route and Bridge (IRB) – VXLAN-EVPN offers significant advantages in Overlay networking by optimizing forwarding decision within the network based on Layer-2 MAC as well as Layer-3 IP information. The decision on forwarding via routing or switching can be done as close as possible to the End-Host, on any given Leaf/ToR (Top-of-Rack) Switch. The Leaf Switch provides the Distributed Anycast Gateway for routing, which acts completely stateless and does not require the exchange of protocol signalization for election or failover decision. All the reachability information available within the BGP control-plane is sufficient to provide the gateway service. The Distributed Anycast Gateway also provides integrated routing and bridging (IRB) decision at the Leaf Switch, which can be extended across a significant number of nodes. All the Leaf Switches host active default gateways for their respective configured subnets; the well known semantic of First Hop Routing Protocols (FHRP) with active/standby does not apply anymore.

Summary – The advantages provided by a VXLAN-EVPN solution are briefly summarized as follows:

  • Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)
  • Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)
  • Forwarding decision based on Control-Plane (minimizes flooding)
  • Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay
  • Leverages Layer-3 ECMP – all links forwarding – in the Underlay
  • Significantly larger Name-Space in the Overlay (16M segments)
  • Integration of Physical and Virtual Networks with Hybrid Overlays
  • It facilitates Software-Defined-Networking (SDN)

Simply formulated, VXLAN-EVPN provides a standards-based Overlay that supports Segmentation, Host Mobility, and High Scale.

VXLAN-EVPN is available on Nexus 9300 (NX-OS 7.0) with  Nexus 7000/7700 (F3 linecards) to follow in the upcoming major release. Additional Data Center Switching platforms, like the Nexus 5600, will follow shortly after.

A detailed whitepaper on this topic is available on In addition, VXLAN-EVPN was featured during the following Cisco Live! Sessions.

Do you have appetite for more? Post a comment, tweet about it and have the conversation going … Thanks for reading and Happy Networking!

Tags: , , , , , , , , , , , ,

How Cisco IT Adopts New Technologies in the Data Center

I’d like to give you an inside look at our Allen Data Center and go over how Cisco IT is adopting new technologies and capabilities while at the same time running the business. I’ll answer your top of mind questions and cover topics such as: Read More »

Tags: , , , , , , , , , , ,