Cisco Blogs

Cisco Blog > Data Center

MDS 9700 Scale Out and Scale Up

This is the final part on the High Performance Data Center Design. We will look at how high performance, high availability and flexibility allows customers to scale up or scale out over time without any disruption to the existing infrastructure.  MDS 9710 capabilities are field proved with the wide adoption and steep ramp within first year of the introduction. Some of the customer use cases regarding MDS 9710 are detailed here. Furthermore Cisco has not only established itself as a strong player in the SAN space with so many industry’s first innovations like VSAN, IVR, FCoE, Unified Ports that we introduced in last 12 years, but also has the leading market share in SAN.

Before we look at some architecture examples lets start with basic tenants any director class switch should support when it coms to scalability and supporting future customer needs

  • Design should be flexible to Scale Up (increase performance) or Scale Out (add more port)
  • The process should not be disruptive to the current installation for cabling, performance impact or downtime
  • The design principals like oversubscription ratio, latency, throughput predictability (as an example from host edge to core) shouldn’t be compromised at port level and fabric level

Lets take a scale out example, where customer wants to increase 16G ports down the road. For this example I have used a core edge design with 4 Edge MDS 9710 and 2 Core MDS 9710. There are 768 hosts at 8Gbps and 640 hosts running at 16Gbps connected to 4 edge MDS 9710 with total of 16 Tbps connectivity. With 8:1 oversubscription ratio from edge to core design requires 2 Tbps edge to core connectivity. The 2 core systems are connected to edge and targets using 128 target ports running at 16Gbps in each direction. The picture below shows the connectivity.

Edge Core Design Day 1

Down the road data center requires 188 more ports running at 16G. These 188 ports are added to the new edge director (or open slots in the existing directors) which is then connected to the core switches with 24 additional edge to core connections. This is repeated with 24 additional 16G targets ports. The fact that this scale up is not disruptive to existing infrastructure is extremely important. In any of the scale out or scale up cases there is minimal impact, if any, on existing chassis layout, data path, cabling, throughput, latency. As an example if customer doesn’t want to string additional cables between the core and edge directors then they can upgrade to higher speed cards (32G FC or 40G FCoE with BiDi ) and get double the bandwidth on the on the existing cable plant.

Edge Core Design Scale UP

Lets look at another example where customer wants to scale up (i.e. increase the performance of the connections). Lets use a edge core edge design for this example. There are 6144 hosts running at 8Gbps distributed over 10 edge MDS 9710s resulting in a total of 49 Tbps edge bandwidth. Lets assume that this data center is using a oversubscription ratio of 16:1 from edge into the core. To satisfy that requirement administrator designed DC with 2 core switches 192 ports each running at 3Tbps. Lets assume at initial design customer connected 768 Storage Ports running at 8G.

Edge Core Design Day1


Few years down the road customer may wants to add additional 6,144 8G ports  and keep the same oversubscription ratios. This has to be implemented in non disruptive manner, without any performance degradation on the existing infrastructure (either in throughput or in latency) and without any constraints regarding protocol, optics and connectivity. In this scenario the host edge connectivity doubles and the edge to core bandwidth increases to 98G. Data Center admin have multiple options for addressing the increase core bandwidth to 6 Tbps.  Data Center admin can choose to add more 16G ports (192 more ports to be precise) or preserve the cabling and use 32G connectivity for host edge to core and core to target edge connectivity on the same chassis. Data Center admin can as easily use the 40G FCoE at that time to meet the bandwidth needs in the core of the network without any forklift. Edge Core Edge Design Scale Out

Or on the other hand customer may wants to upgrade to 16G connectivity on hosts and follow the same oversubscription ratios. . For 16G connectivity the host edge bandwidth increases to 98G and data center administrator has the same flexibility regarding protocol, cabling and speeds.

Edge Core Edge Example 1 ScaleUP

For either option the disruption is minimal. In real life there will be mix of requirements on the same fabric some scale out and some scale up. In those circumstances data center admins have the same flexibility and options. With chassis life of more than a decade it allows customers to upgrade to higher speeds when they need to without disruption and with maximum flexibility. The figure below shows how easily customers can Scale UP or Scale Out.


Edge Core Edge Design Scale Out Scale Up


As these examples show Cisco MDS solution provides ability for customers to Scale Up or Scale out in flexible, non disruptive way.

“Good design doesn’t date. Bad design does.”
Paul Rand


Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Enabling Data Center Services with RISE : Remote Integrated Services Engine

Data Centers are becoming increasingly smart, intelligent and elastic. With the advancement in cloud and virtualization technologies, customers demand dynamic workload management, efficient and optimal use of their resources. In addition the configuration and administration of Data Center solutions is complex and is going to become increasingly so.RISE

With these requirements and architectures in mind we have a industry first solution called Remote Integrated Service Engine (RISE).  RISE is a technology that simplifies provisioning, out of box management of service appliances like load balancers, firewalls, network analysis modules. It makes data center and campus networks dynamic, flexible, easy to configure and maintain.

RISE can dynamically provision network resources for any type of service appliance (physical and virtual form factors). External appliances can now operate as integrated service modules with Nexus Series of switches without burning a  slot in a switch . This technology provides robust application delivery capabilities that accelerate the application performance manifold.

RISE is supported on all Nexus Series switches with services like Citrix NetScaler MPX, VPX, SDX and Cisco Prime NAM with many more in the pipeline.

Advantages & Features

  1. Simplified Out-of-Box experience : reduces the administrator’s manual configuration steps from 30 to 8 steps !!
  2. Supported on Citrix NetScaler MPX, SDX, VPX, and Nexus 1KV with VPX
  3. Supported on Cisco Prime Network Analyzer Module
  4. Automatic Policy Based Routing – Eliminates need for SNAT or Manual PBR
  5. Direct and Indirect Attach mode integration
  6. Show module for RISE
  7. Attach module for RISE
  8. Auto Attach – Zero touch configuration of RISE
  9. Health Monitoring of appliance
  10. Appliance HA and VPC supported
  11.  Nexus 5K/6K support (EFT available)
  12. IPV6 support (EFT available)
  13. DCNM support
  14. Order of magnitude OPEX savings: reduction in configuration, and ease of deployment
  15. Order of magnitude CAPEX savings: Wiring, Power Rackspace and Cost savings

For more information, schedule an EFT or POC Contact us at


RISE press release on Wall Street Journal :
RISE At A Glance white paper:
RISE Video at Interop:
Cisco RISE page:
Gartner blog on RISE: “Cisco and Citrix RISE to the Occasion”:

Tags: , , , , , , , , , , , , ,

Next Generation Data Center Design With MDS 9700 – Part III

Note: This is the third of a three-part series on Next Generation Data Center Design with MDS 9700; learn how customers can deploy scalable SAN networks that allow them to Scale Up or Scale Out in a non disruptive way.  Part 1 | Part 2 ]

This week is exciting, had opportunity to sit on round table with Cisco’s largest customers on an open ended architecture discussion and their take on past, present and future. More on that some other time let’s pick up last critical aspect of High Performance Data Center design namely flexibility. Customers need flexibility to adapt to changing requirements over time as well as to support diverse requirements of their users. Flexibility is not just about protocol, although protocol is very important aspect, but it is also about making sure customers have choice to design, grow and adapt their DC according to their needs. As an example if customers want to utilize the time to market advantage and ubiquity of Ethernet they can by adopt FCoE.


Moreover flexibility has to be complemented by seamless integration where customers can not only mix and match the architectures/protocols/speeds but also evolve from one to other over time with minimal disruption and without forklift upgrades. Investment protection of more than a decade on Cisco director switches allows customer to move to higher speeds, or adopt new protocols using the existing chassis and fabric cards. Finally any solution should allow scalability over time with minimal disruptions and common management model. As an example on MDS 9710 or MDS 9706 customers can choose to use 2/4/8 G FC, 4/8/16G FC, 10G FC or 10G FCoE at each hop.

Multiprotocol Innovation

Let’s review each aspect of flexibility at a time.



Cisco SAN product family is designed to support Architecture flexibility. From smallest to  the largest customers and everything in-between.  Customers can grow from 12 16G ports to 48 ports on a single 9148S. They can grow from 48 16G Line Rate Ports to 192 16G Line Rate with MDS 9710 and upto 384 ports on MDS 9710. Finally having seamless FC and FCoE capability allows customers to use these directors as edge or core switches . With the industry leading scalability numbers, customers can scale up or scale out as per their needs. Two examples show how customers can use Director class switches (9513, 9506, 9710 or 9706) based Architecture for End of Row designs. Similarly customers can orchestrate Top of Rack designs using Nexus fixed family or MDS 9148S.

Examples of Edge Core Designs with MDS ToR and EoR

If they want to continue with FC for foreseeable future or have sizable FC infrastructure that they want to leverage (and have option to go to FCOE) then MDS serves their needs. Similarly they can support edge core designs, and edge core edge designs or even collapsed  cores if so desired.


FC Edge Core and Edge Core Edge


If customers need converged switch then Nexus 2K, 5K and 6K provides the flexibility, ability to collapse two networks, simplify management as shown in the picture below.

FEX and Nexus Edge Design Options


Customers can mix and match the FC speeds 2G/4G/8G, 4G/8G/16G on the latest MDS 9148S, and MDS 9700 product family. With all the major optics supported, customers can pick and choose optics for the smallest distance to long distance CWDM and DWDM solutions in addition to SW, LW and ER optics choices. In addition MDS 9700 supports 10GE optics running 10G FC traffic for ease of implementing 10G DWDM solutions based on ubiquitous 10GE circuits.


FC is a dominant protocol with DC but at the same time a lot of customers are adopting FCoE to improve ROI, simplify the network or simply to have higher speeds and agility. Irrespective of the needs and timeline MDS solution allows customer to adopt FCoE today or down the road without forklift upgrades on the existing MDS 9700 platforms while leveraging the existing FC install base.

FCoE Flexibility

The diagram above shows how customers can collapse LAN and SAN networks on the edge into one network. The advantage of FEX include reduced TCO, simplified operations (Parent switch provides a single point of management and policy enforcement and Plug-and-play management includes auto-configuration).

Another example to allow non transition less disruptive for customers Cisco has supported the BiDi optics on the Nexus product family. This allows customers to use the the same same OM2, OM3 and OM4 fabrics for 40G FCoE connectivity and still don;t have to rip and replace cabling plant.

BiDi Option

For customer who are not ready to converge networks but want to achieve faster time to market, higher performance, Ethernet scale economies can use separate LAN and SAN network and use FCoE for that dedicated SAN .

Evolution path from FC to FCoE

Coupled with broad Cisco product portfolio means that customers have the maximum flexibility to tune the architecture precisely to their needs. Cisco product portfolio is tightly integrated, all the SAN switches use same NxOS and DCNM provides seamless manageability across LAN, SAN, Converged infrastructure to Fabric Interconnects on UCS.

Broad Product Portfolio

From the last 3 blogs lets quickly capture what are the unique characteristics of MDS 9700 that allows for High Performance Scalable Data Center Design.

24 Tbps Switching capacity, line rate 16g FC ports, No Oversubscription, local switching or bandwidth allocation.

Redundancy for every critical component in the chassis including Fabric Card. Data Resiliency with CRC check and Forward Error Correction. Multiple level of CRC checks, smaller failure domains.

In next few days lets put this all together to see how customers can deploy scalable networks that allow them to Scale Up or Scale Out in a non disruptive way.

To learn more about the MDS 9148S please join us for a webinar.

“In business, words are words; explanations are explanations, promises are promises, but only performance is reality.”

Harold S. Geneen

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Integrating ACI & Nexus 2000-7000

As the breadth and depth of the ACI solution continues to grow, so does customer interest.  Many customers who have invested in, and continue to invest in, the Nexus 2000-7000 switches find the ACI vision very compelling.  So, this leads to a logical question regarding how an existing Nexus 2000-7000 fabric will integrate with an ACI fabric.

In short, customers can leverage current Nexus products and add ACI capabilities to their data centers in an incremental manner.  Integrating ACI into an existing Nexus environment will not require replacement of existing Nexus switches.  The benefits of ACI policy can be extended to apps on both physical and virtual servers within the existing Nexus fabric.  This can be achieved as follows (double click on the graphic below to launch the 3+ minute presentation):

In this scenario, the existing Nexus fabric is serving as an optimized transport for an ACI overlay solution.  However, this solution is very different from other industry overlay solutions.   It’s different in that the ACI overlay provides integrated/embedded support for both physical and virtual servers, it allows use of existing L4-7 infrastructure, while providing the automation functionality of the ACI policy model.

If you’d like to learn more, there is a summary, as well as a white paper available.  There is also this video whiteboard session that covers a subset of the elements mentioned above.

Tags: , , , , , ,

Meet the Next Generation Fabric Extender – Cisco’s Nexus 2300 Series

More than 5 years ago, Cisco introduced its first Nexus 2000 Fabric Extender (FEX) into the market. This broad portfolio has enabled over 15,000 customers to seamlessly add network access port density across server racks without adding the inherent management complexity that comes with adding more boxes into your architecture to meet scaling requirements which ultimately translates into lower capex and opex for the business.

Today marks the next evolution for this portfolio as we introduce the Nexus 2300 platform – the 3rd generation Fabric Extender family. Based on the extensive innovations you have come to know from the Nexus 2100 and 2200 platforms, Nexus 2300 Fabric Extenders expand on these capabilities with:

  • Larger buffers to absorb bursts of traffic for a wide variety of workloads such as multicast feeds, voice traffic, video traffic, and healthcare applications
  • Unified Ports support enabling a flexible LAN and SAN deployment through support for Ethernet, Fiber Channel and Fiber Channel over Ethernet connectivity
  • Support for Cisco’s 40G BiDi optics simplifying migration 10 to 40 Gigabit Ethernet speeds while reusing existing 10G cabling
  • Additional versatile TCAM which can be used for:
    • Advanced features such as ACL classifications and QOS
    • Hardware-capable local flow redirect for architectures that require intra-rack traffic to reduce bandwidth

Put all of these together with the single point of management, policy enforcement, zero-touch provisioning installation and automatic configuration that is available across all Nexus 2000 Fabric Extenders and you further benefit from a more simplified and flexible network design that helps you commission and decommission server racks faster, simplify operations, and support varying workload requirements.

Now let’s look at the first member of the Nexus 2300 platform, the Nexus 2348UPQ. Priced at $9500 (US List), almost the same price as current 1Gbps FEXs, the Nexus 2348UPQ supports 48 1/10 Gigabit Ethernet ports. You can add further 10 G ports when you split the 40G ports for up to 64 10G ports – perfect for data centers that are migrating their servers from 1 to 10G network connectivity. The 6 on-board 40Gbps ports support Cisco’s BiDi optics so that as you migrate from 10G to 40G as your uplink speed, you can reuse your existing 10G cabling – helping you save on the costs of re-cabling your network and get it upgraded faster! This new fabric extender can be deployed in conjunction with Nexus 5500, 5600, and 6000 parent switches and with the Nexus 7000 and 9000 Series in the future.

In summary, Cisco’s Fabric Extender portfolio has seen tremendous traction in the market as the many benefits of this architecture ultimately help create a cost-effective, flexible and simplified approach to building a data center network. The Nexus 2300 Series Fabric Extender, with the Nexus 2348UPQ, continues to deliver on these same principles and further expands on the promise of helping simplify network deployment and operations while ensuring the data center network is ready to support varying application needs. I invite you to learn more about the Nexus 2348UPQ and other Nexus 2000 Series Fabric Extenders at

Tags: , , , , , ,