Cisco Blogs


Cisco Blog > Data Center and Cloud

MDS 9700 Scale Out and Scale Up

This is the final part on the High Performance Data Center Design. We will look at how high performance, high availability and flexibility allows customers to scale up or scale out over time without any disruption to the existing infrastructure.  MDS 9710 capabilities are field proved with the wide adoption and steep ramp within first year of the introduction. Some of the customer use cases regarding MDS 9710 are detailed here. Furthermore Cisco has not only established itself as a strong player in the SAN space with so many industry’s first innovations like VSAN, IVR, FCoE, Unified Ports that we introduced in last 12 years, but also has the leading market share in SAN.

Before we look at some architecture examples lets start with basic tenants any director class switch should support when it coms to scalability and supporting future customer needs

  • Design should be flexible to Scale Up (increase performance) or Scale Out (add more port)
  • The process should not be disruptive to the current installation for cabling, performance impact or downtime
  • The design principals like oversubscription ratio, latency, throughput predictability (as an example from host edge to core) shouldn’t be compromised at port level and fabric level

Lets take a scale out example, where customer wants to increase 16G ports down the road. For this example I have used a core edge design with 4 Edge MDS 9710 and 2 Core MDS 9710. There are 768 hosts at 8Gbps and 640 hosts running at 16Gbps connected to 4 edge MDS 9710 with total of 16 Tbps connectivity. With 8:1 oversubscription ratio from edge to core design requires 2 Tbps edge to core connectivity. The 2 core systems are connected to edge and targets using 128 target ports running at 16Gbps in each direction. The picture below shows the connectivity.

Edge Core Design Day 1

Down the road data center requires 188 more ports running at 16G. These 188 ports are added to the new edge director (or open slots in the existing directors) which is then connected to the core switches with 24 additional edge to core connections. This is repeated with 24 additional 16G targets ports. The fact that this scale up is not disruptive to existing infrastructure is extremely important. In any of the scale out or scale up cases there is minimal impact, if any, on existing chassis layout, data path, cabling, throughput, latency. As an example if customer doesn’t want to string additional cables between the core and edge directors then they can upgrade to higher speed cards (32G FC or 40G FCoE with BiDi ) and get double the bandwidth on the on the existing cable plant.

Edge Core Design Scale UP

Lets look at another example where customer wants to scale up (i.e. increase the performance of the connections). Lets use a edge core edge design for this example. There are 6144 hosts running at 8Gbps distributed over 10 edge MDS 9710s resulting in a total of 49 Tbps edge bandwidth. Lets assume that this data center is using a oversubscription ratio of 16:1 from edge into the core. To satisfy that requirement administrator designed DC with 2 core switches 192 ports each running at 3Tbps. Lets assume at initial design customer connected 768 Storage Ports running at 8G.

Edge Core Design Day1

 

Few years down the road customer may wants to add additional 6,144 8G ports  and keep the same oversubscription ratios. This has to be implemented in non disruptive manner, without any performance degradation on the existing infrastructure (either in throughput or in latency) and without any constraints regarding protocol, optics and connectivity. In this scenario the host edge connectivity doubles and the edge to core bandwidth increases to 98G. Data Center admin have multiple options for addressing the increase core bandwidth to 6 Tbps.  Data Center admin can choose to add more 16G ports (192 more ports to be precise) or preserve the cabling and use 32G connectivity for host edge to core and core to target edge connectivity on the same chassis. Data Center admin can as easily use the 40G FCoE at that time to meet the bandwidth needs in the core of the network without any forklift. Edge Core Edge Design Scale Out

Or on the other hand customer may wants to upgrade to 16G connectivity on hosts and follow the same oversubscription ratios. . For 16G connectivity the host edge bandwidth increases to 98G and data center administrator has the same flexibility regarding protocol, cabling and speeds.

Edge Core Edge Example 1 ScaleUP

For either option the disruption is minimal. In real life there will be mix of requirements on the same fabric some scale out and some scale up. In those circumstances data center admins have the same flexibility and options. With chassis life of more than a decade it allows customers to upgrade to higher speeds when they need to without disruption and with maximum flexibility. The figure below shows how easily customers can Scale UP or Scale Out.

 

Edge Core Edge Design Scale Out Scale Up

 

As these examples show Cisco MDS solution provides ability for customers to Scale Up or Scale out in flexible, non disruptive way.

“Good design doesn’t date. Bad design does.”
Paul Rand

 

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Next Generation Data Center Design With MDS 9700 – Part III

This week is exciting, had opportunity to sit on round table with Cisco’s largest customers on an open ended architecture discussion and their take on past, present and future. More on that some other time let’s pick up last critical aspect of High Performance Data Center design namely flexibility. Customers need flexibility to adapt to changing requirements over time as well as to support diverse requirements of their users. Flexibility is not just about protocol, although protocol is very important aspect, but it is also about making sure customers have choice to design, grow and adapt their DC according to their needs. As an example if customers want to utilize the time to market advantage and ubiquity of Ethernet they can by adopt FCoE.

Flexibility

Moreover flexibility has to be complemented by seamless integration where customers can not only mix and match the architectures/protocols/speeds but also evolve from one to other over time with minimal disruption and without forklift upgrades. Investment protection of more than a decade on Cisco director switches allows customer to move to higher speeds, or adopt new protocols using the existing chassis and fabric cards. Finally any solution should allow scalability over time with minimal disruptions and common management model. As an example on MDS 9710 or MDS 9706 customers can choose to use 2/4/8 G FC, 4/8/16G FC, 10G FC or 10G FCoE at each hop.

Multiprotocol Innovation

Let’s review each aspect of flexibility at a time.

 

Architecture:

Cisco SAN product family is designed to support Architecture flexibility. From smallest to  the largest customers and everything in-between.  Customers can grow from 12 16G ports to 48 ports on a single 9148S. They can grow from 48 16G Line Rate Ports to 192 16G Line Rate with MDS 9710 and upto 384 ports on MDS 9710. Finally having seamless FC and FCoE capability allows customers to use these directors as edge or core switches . With the industry leading scalability numbers, customers can scale up or scale out as per their needs. Two examples show how customers can use Director class switches (9513, 9506, 9710 or 9706) based Architecture for End of Row designs. Similarly customers can orchestrate Top of Rack designs using Nexus fixed family or MDS 9148S.

Examples of Edge Core Designs with MDS ToR and EoR

If they want to continue with FC for foreseeable future or have sizable FC infrastructure that they want to leverage (and have option to go to FCOE) then MDS serves their needs. Similarly they can support edge core designs, and edge core edge designs or even collapsed  cores if so desired.

 

FC Edge Core and Edge Core Edge

 

If customers need converged switch then Nexus 2K, 5K and 6K provides the flexibility, ability to collapse two networks, simplify management as shown in the picture below.

FEX and Nexus Edge Design Options

Speeds

Customers can mix and match the FC speeds 2G/4G/8G, 4G/8G/16G on the latest MDS 9148S, and MDS 9700 product family. With all the major optics supported, customers can pick and choose optics for the smallest distance to long distance CWDM and DWDM solutions in addition to SW, LW and ER optics choices. In addition MDS 9700 supports 10GE optics running 10G FC traffic for ease of implementing 10G DWDM solutions based on ubiquitous 10GE circuits.

Protocol

FC is a dominant protocol with DC but at the same time a lot of customers are adopting FCoE to improve ROI, simplify the network or simply to have higher speeds and agility. Irrespective of the needs and timeline MDS solution allows customer to adopt FCoE today or down the road without forklift upgrades on the existing MDS 9700 platforms while leveraging the existing FC install base.

FCoE Flexibility

The diagram above shows how customers can collapse LAN and SAN networks on the edge into one network. The advantage of FEX include reduced TCO, simplified operations (Parent switch provides a single point of management and policy enforcement and Plug-and-play management includes auto-configuration).

Another example to allow non transition less disruptive for customers Cisco has supported the BiDi optics on the Nexus product family. This allows customers to use the the same same OM2, OM3 and OM4 fabrics for 40G FCoE connectivity and still don;t have to rip and replace cabling plant.

BiDi Option

For customer who are not ready to converge networks but want to achieve faster time to market, higher performance, Ethernet scale economies can use separate LAN and SAN network and use FCoE for that dedicated SAN .

Evolution path from FC to FCoE

Coupled with broad Cisco product portfolio means that customers have the maximum flexibility to tune the architecture precisely to their needs. Cisco product portfolio is tightly integrated, all the SAN switches use same NxOS and DCNM provides seamless manageability across LAN, SAN, Converged infrastructure to Fabric Interconnects on UCS.

Broad Product Portfolio

From the last 3 blogs lets quickly capture what are the unique characteristics of MDS 9700 that allows for High Performance Scalable Data Center Design.

24 Tbps Switching capacity, line rate 16g FC ports, No Oversubscription, local switching or bandwidth allocation.

Redundancy for every critical component in the chassis including Fabric Card. Data Resiliency with CRC check and Forward Error Correction. Multiple level of CRC checks, smaller failure domains.

In next few days lets put this all together to see how customers can deploy scalable networks that allow them to Scale Up or Scale Out in a non disruptive way.

To learn more about the MDS 9148S please join us for a webinar.

“In business, words are words; explanations are explanations, promises are promises, but only performance is reality.”

Harold S. Geneen

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

EVRY protects SAN investment, expands Capabilities

EVRY chooses Cisco MDS 9710 to adapt easily to emerging and future customer demands. EVRY provides IT services to businesses, financial institutions, national public sector entities, municipalities, and health authorities in Norway and other Nordic countries.

EVRY’s virtualization deployments are expanding rapidly and customers deploying more applications placed higher demands on storage environments. EVRY’s requirements include 16-Gbps Fibre Channel connectivity without oversubscription, high reliability storage environment and the system had to support switching for 12 petabytes of storage on more than 100 storage arrays. Of course it had to be easy to manage as they have a very small team.

EVRY chose Cisco MDS 9710 Multilayer Director switches for their SAN core. The new systems were delivered and supported by Cisco storage partner EMC.

Read More »

Tags: , , ,

Cisco MDS for Small to Cloud-Scale Data Centers

Superior Platforms, Scale, and Operational Simplicity

 Data Center trends like  Virtualization, Solid State Drives, DC consolidation and Data Explosion are putting a tremendous amount of strain on the infrastructure. These challenges need targeted and multifaceted approach. It has to be holistic solution to the problems rather than point products for each unique problem. Data Centers require improvements in performance, flexibility, scalability, and reliability and ease of management. To address that Cisco revamped the MDS product line, the journey we started last year when we introduced 9710 and 9250i.

9710 – Director Class Switch with 3x the performance of any director, 384 ports of line rate 16G FC, highest reliability and flexibility.

9250i – Services Appliance supporting 10G FCIP, 16G FC and 10G FCoE in addition to IO Acceleration, Data Mobility Migration in compact 2 RU form factor.

We had a great success with the product line with steepest ramp and amazing customer feedback. Building on the success we have added new members to the product family and extended the innovation to allow for simpler management and scalable deployments.

Portfolio

 

a) Three New Products

  1. MDS 9148S – Industries’  most versatile affordable 1RU switch with High-Performance, Easy of Deployment,  Enterprise-class features
  2. MDS 9706 -- Unprecedented investment protection with high performance, reliability and multi-protocol flexibility
  3. High Density line rate 10G FCoE Card -- For customers to adopt high density FCoE in incremental non-disruptive fashion on the existing FC footprint without forklift upgrades.

b)     New Scalable Deployment Options

  1. Much Higher Scalability for SAN Infrastructures.
  2. Dynamic FCoE over Fabric Path
  3. Data migration enhancements for speed, scale and resiliency

c)     New Management Features

  1. Hardware based FC Congestion Detection and Recovery
  2. Integration with Industry leading Platforms
  3. End to End Visibility
  4. Switch Health Score

With the addition of new members Cisco not only has best of the breed products but also broadest product portfolio. This allows customers to design the SAN precisely to their needs from small departmental SANs to the largest enterprises, from traditional LAN, SAN networks to fully converged fabric and everything in between.

Product Portfolio

Attend the Webinar on August 12th 8:00 PST to learn more :  Register Now

 

Lets look at the capabilities of each product in little more detail

  Cisco MDS 9148S: High-Performance, Easy to Deploy, Enterprise-class Fabric Switch

Cisco MDS 9148S Fixed Fabric Switch

Versatile: 9148S pay as you grow model allows customers to start from small base and grow . It allows customers to grow from 12 Ports to 24, 36 and finally 48 without any rip and replace. It allows customers to go from 2/4/8G to 16G FC speeds. It is not only the most affordable switch shipping today across all the possible configurations but with 2x the range of ports it allows unparalleled scalability for future growth.

Ease of use: Power On Auto provisioning which allows   9148 and 9148S to automate switch setup. From getting DHCP, to downloading and applying the software to the final configuration is done automatically. Quick configuration wizard allows the box to be configured in an easy way. It shares the same NxOS as rest of the MDS and Nexus products. Power on Auto Provisioning (POAP) is important for large scale data centers where 9148S will be used as Top of the Rack (ToR) switch and distributed throughout the data center. This saves customer to go from box to box with the serial cable and program them individually. It allows for  rapid, error free and consistent provisioning.

9148S ToR Architecture

MDS 9148 Pay As You Grow

Enterprise Class switch: It offers the rich Enterprise features like non-disruptive software upgrade,32 Virtual SANs (VSANs), Inter-VSAN Routing (IVR), QOS, PortChannels, N-Port ID Virtualization (NPIV), N-Port Virtualization (NPV), Comprehensive Security in addition to redundant power supplies and fans. Its first of the kind switch in the industry to allow hardware based slow drain detection and recovery. It has back to front airflow.

Customer Use Case: Customer will use the 9148S to design small SAN environments like  departmental SANs.  Larger Enterprises will use 9148S as ToR Switch for ease of cabling and ease of Management. In addition to that 9148S will be used for BC/DR or remote locations. Pay as you go model is very attractive to customers as it allows them to grow the port count from 12 to 48 without any price penalty as their network demands grow.

 Cisco MDS 9706: Extending MDS 9710 Director Qualities to a Smaller Form Factor

Picture4

It is the highest performance director in the industry. It provides 3X the bandwidth compared to any compact director in the industry. Not only it provide 192 ports line rate performance at 16G but it is designed to provide line rate performance at 32G FC and 40G FCoE when those line cards are introduced without the forklift upgrades using the same type of fabric cards. With 6 fabric cards it provides 1.5Tbps of bandwidth per slot.

In addition to that this is industries first class of directors to offer Redundancy on all critical components including fabric cards. Smaller failure domain, Forward error correction, multi-point CRC checks, predictable and consistent performance for both latency and throughput.

Small to medium enterprises will use 9706 as Middle of the row and end of row switch in addition line rate 16G performance allows it to be used for connectivity to targets in addition to host connectivity. It will be used for both edge core and edge-core-edge designs.

In addition to the pod like deployments where 9RU form factor and 192 ports of line rate at 16G is very attractive.

Some of the specs of the switch are enumerated below

 

  • 1.5 Tbps per slot switching capability

 192 ports of 16G FC line rate today with 100% head room to grow to 32G FC) without forklift upgrade

  • Industries Highest ReliabilityMDS 9706 Details

N+1 Fabric redundancy, smaller failure domains, Forward Error Correction, CRC error checks at multiple points, In service software upgrades,  Crossbar design with central arbitration and Virtual Output Queuing ensure customers not only get highest availability but also predictable and consistent throughput independent of the traffic profile.

  • Unmatched Flexibility

With ability to support both FC and FCoE line cards. With capability to support 2/4/8/10/16G FC and 10G FCoE today and performance to support 32G FC and 40G FCoE on the same footprint.

Industry’s Highest-Density FCoE Module on a FC Director

With 48 ports this has the highest port density and greatest flexibility in the industry. Without any restrictions Cisco customers can now orchestrate FC, FCoE and mixed solutions. FCoE line card afford customers ability to design FC solutions and incrementally deploy FCoE without forklift upgrades and meeting the same features, reliability and availability as afforded by FC.

FCoE Line Card

In addition to hardware we added extensive capabilities to enable small size to cloud scale deployments.

Scalability Enhancements

To support large scale out and scale up deployment models we have increased the scalability limits for the SAN infrastructure. The industry leading scalability numbers allow Cisco customer’s unprecedented future proofing and scalability to Scale out or Scale-up. Finally the Data Mobility Migration has 2x the speed and 8x the scale and higher resiliency.

Simplifying SAN Management

In addition to enhanced capabilities in Cisco tools MDS family is integrated with industry standard tools to provide faster configurations like automated zoning. Some of the examples of the tools are UCS Director, EMC ViPR, Microsoft System Center VMM and IBM PpowerVC.

To address complexity in the data-centers Cisco is focused on SAN Management simplification. First and foremost that is top of mind for customers is slow drain. If there are slow draining devices in the network it chokes the entire fabric. These conditions are transient, extremely difficult to isolate, debug and fix. To detect and recover for these conditions Cisco introduced Slow Drain Detection and Recovery in software in previous generation of devices. Now with the new products we have provided the support for these devices to run the slow drain detection and recovery in the hardware rather than waiting for software to come around polling individual ports every 100 ms which is a life time in the data center. As the table below shows with hardware based slow drain the detection speed has increased 100 times and recovery is of the order of nano seconds rather than 100ms.

 

Platforms Detection Granularity Recovery Action Latency – Start   and Stop
Software-based MDS 9500MDS 9148 100ms 100ms
Hardware-based   NEW! MDS 9700,   MDS   9250i, MDS 9148S 1ms Immediate   (ns)

 

For more info read this whitepaper

 End to End Visibility

In addition to that Data Center Network Manager (DCNM) provides end to end visibility from hosts (virtual or Physical) through switches (MDS or Nexus) into the storage arrays independent of the protocol. DCNM is single pane of glass visibility into the Data Center for both SAN and LAN.

HPRA

Host Path Redundancy Analysis checks the network every 24 hours or customer designated interval if there is end to end dual paths from Host to the target. It checks against port down situations, VSAN mismatches, VSAN Segmentation, LUN mismatches as well as makes sure both the ports are not on the same line cards. Similar activity that used to take months is now completed on the fly every 24 hours reducing risk and time to repair. Further more administrators are not surprised by an outage as they have complete visibility for the dual paths. Furthermore having both the paths up allows to mitigate any silent failures as well as avoid outages if one of the SAN fails.

Switch health score is another unique capability of DCNM  to track switch health over time. It allows customer to quickly determine level of risk, isolate and fix the alerts resulting in low health score and track the health of the SAN over time.

 

Health Score

 As I started the discussion today Data Centers need a holistic approach to solving the challenges of the data center. Customers not only need higher performance, investment protection, lower opex and capex, reliability but also ease of management and tightly integrated end to end solution. The solutions and capabilities I described allows us to solve the challenges faced by data centers not only today for the years to come. We introduced MDS products in 2002 and since then we have introduced industries first innovation, just few examples out of that are enumerated below. We will continue to innovate in this space for the next decade.

 

Continued Innovation

 

Prashant Jain

Sr. Product Manager, DCBU

“The best time to plant a tree was 20 years ago. The second best time is now”

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Next Generation Data Center Design With MDS 9710 – Part II

EMC World was wonderful. It was gratifying to meet industry professionals,  listen in on great presentations and watch the demos for key business enabling technologies that Cisco, EMC and others have brought to fruition.  Its fascinating to see the transition of DC from cost center to a strategic business driver . The same repeated all over again at Cisco Live. More than 25000 attendees, hundreds of demos and sessions. Lot of  interesting customer meetings and MDS continues to resonate. We are excited about the MDS hardware that was on the display on show floor and interesting Multiprotocol demo and a lot of interesting SAN sessions.

Outside these we recently did a webinar on how Cisco MDS 9710 is enabling High Performance DC design with customer case studies. You can listen to that here.

Three Pillars of ReliabilitySo let’s continue our discussion. There is no doubt when it comes to High Performance SAN switches there is no comparable to Cisco MDS 9710. Another component that is paramount to a good data center design is high availability. Massive virtualization, DC consolidation and ability to deploy more and more applications on powerful multi core CPUs has increased the risk profile within DC. These DC trends requires renewed focus on availability. MDS 9710 is leading the innovation there again. Hardware design and architecture has to guarantee high availability. At the same time, it’s not just about hardware but it’s a holistic approach with hardware, software, management and right architecture. Let me give you some just few examples of the first three pillars for high reliability and availability.

 

Reliability examples in MDS

 

Picture6

MDS 9710 is the only director in the industry that provides Hardware Redundancy on all critical components of the switch, including fabric cards. Cisco Director Switches provide not only CRC checks but ability to drop corrupted frames. Without that ability network infrastructure exposes the end devices to the corrupted frames. Having ability to drop the CRC frames and quickly isolate the failing links outside as well as inside of the director provides Data Integrity and fault resiliency. VSAN allows fault isolation, Port Channel provides smaller failure domains, DCNM provides rich feature set for higher availability and redundancy. All of these are but a subset of examples which provides high resiliency and reliability.

 

Weakest link

 

We are proud of the 9500 family and strong foundation for reliability and availability that we stand on. We have taken that to a completely new level with 9710. For any design within Data center high availability  has to go hand in hand with consistent performance. One without the other doesn’t make sense. Right design and architecture with DC as is important as components that power the connectivity. As an example Cisco recommend customers to distribute the ISL ports of an Port Channel across multiple line cards and multiple ASICs. This spreads the failure domain such that any ASIC  or even line card failures will not impact the port channel connectivity between switches and no need to reinitiate all the hosts logins. You can see white paper on Next generation Cisco MDS here. At part of writing this white paper ESG tested the Fabric Card redundancy (Page 9) in addition to other features of the platform. Remember that a chain is only as strong as its weakest link.

 

Geschäftsmann hat Wut, Frust und Ärger im Büro

 

The most important aspect for all of this is for customer is to be educated.

Ask the right questions. Have in depth discussions to achieve higher availability and consistent performance. Most importantly selecting the right equipment, right architecture and best practices means no surprises.

We will continue our discussion for the Flexibility aspect of MDS 9710.

 

 

-We are what we repeatedly do. Excellence, then, is not an act, but a habit (Aristotle)

 

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,