How to get more SAN mileage out of UCS FI?

 

Mileage (miles per gallon) is one of the important criteria while buying any automobile and once bought, it is highly desirable to hit the maximum advertised mileage without significantly changing the driving habits or the routes (highway vs city mpg). Well, I have not been able to achieve that yet, so being a geek, I focused my attention on a different form of mileage (throughput per switch-port) that interests me at work. So in this blog, I would explore a way to get more SAN mileage from the Cisco UCS FI (Fabric Interconnect) without significantly affecting the SAN admin’s day-to-day operations.

Context:

Just a bit of background before we delve into the details – The I/O fabric between the UCS FI and the UCS Blade Server Chassis is a converged fabric, running FCoE. The usage of FCoE within the UCS fabric is completely transparent to the host operating system, and any Fibre Channel block storage traffic traverses this fabric as the FCoE traffic. So, a large number of over 20,000+ UCS customers, using Block Storage, are already using FCoE at the access layer of the network.

Choices:

Now, the key question is what technology, FC or FCoE, to use northbound on the FI uplink ports to connect to an upstream Core switch for the SAN connectivity. So, what are the uplink options? Well, the FI has Unified ports and the choice is using the same uplink port as either 8G FC -or- 10G FCoE. [Note that when using the FCoE uplink, it is not a requirement to use a converged link and one can still use a dedicated FCoE link for carrying pure SAN traffic].

Observations:

1)    Bandwidth for Core Links: This is a very important aspect for the core part of the network. It is interesting to note that 10G FCoE provides almost 50% more throughput than the 8G FC. This is because FC has a different bit encoding and clock-rate than Ethernet, and so 8G FC yields 6.8G throughput while 10G FCoE yields close to 10G throughput (post 1-2% Ethernet frame overhead)

2)   Consistent Management Model: FCoE is FC technology with same management and security model, so it will be a seamless transition for a SAN admin to move from FC to FCoE with very minimal change in the day-to-day operations. Moreover, this FCoE link is carrying dedicated SAN traffic without requiring any convergence of LAN traffic. To add to that, if the UCS FI is running in the NPV mode, then technically the FCoE link between the UCS FI and the upstream SAN switch does not constitute a Multi-Hop FCoE design, as the UCS FI is not consuming a Domain-ID, and the bulk of SAN configurations like zoning etc. need to happen on only the Core SAN switch, thus maintaining the same consistent SAN operational model as with just the FC.

3)    Investment Protection with Multi-protocol flexibility: By choosing FCoE uplink from the converged access layer, one can still continue to use the upstream  core SAN Director switch as-is, providing the connectivity to existing FC Storage arrays. Note that Cisco MDS 9000 SAN Director offers Multi-protocol flexibility so that one can Interconnect FCoE SANs on the Server-side with the FC SANs on the Storage-side.

And, we have a winner…

Based on the afore-mentioned observations, the needle moves in the favor of using the 10G FCoE technology as one can now get significantly higher SAN mileage from the same uplink port on the UCS FI without significantly affecting the operational model!

Furthermore, once comfortable with FCoE technology on a separate SAN Director switch, the customers can now choose to consolidate the upstream LAN and SAN Core switch as the Unified Director-Class switch, such as the Cisco Nexus 7000, while still maintaining dedicated LAN and SAN links. Note that one can create a Storage VDC (Virtual Device Context), which behaves like a “virtual MDS” on the Nexus 7000 and thus provides management and fault isolation between the LAN and SAN pieces. If it is a Greenfield deployment with the FCoE Storage Arrays, then those could be deployed directly off the Nexus 7000. This architecture would also provide utmost agility as one can now run any other Storage protocol, be it iSCSI or NFS, as well at any time on this infrastructure.

So, the initial choice of using FCoE from the UCS FI could provide one the path of future-proofing the SAN. Note that I have taken the example of UCS FI in this blog, however, any other converged access switches available today, present similar uplink options – 8G FC or 10G FCoE. And the choice of getting the appropriate SAN mileage is yours!