Cisco Blogs

Is The Fibre Channel Killer Born, yet?

March 30, 2007 - 4 Comments

There has been a lot of discussion on various online forums lately about the future of Fibre Channel. The piece that probably got the attention of most of us was the extremely well-written article published by Deni Connor on Network World.I have waited a few days before posting anything as Cisco clearly has a significant stake in Fibre Channel as well as iSCSI and Ethernet technologies for the Data Center in general, not to mention our solutions for InfiniBand switching (all right, this is the last piece of sales on this posting!)The life or death of Fibre Channel is clearly something we care about.Aside from emotions or personal preferences, I think nobody can argue with the fact that Fibre Channel is alive and kicking. The Fibre Channel market continues to grow and all of the major players in this market (including Cisco) continue to invest aggressively in Fibre Channel R&D and sales. And this is not because we want it, but because we see a lot of demand for it. iSCSI is a valid alternative to Fibre Channel in certain environments, but it still does not represent a perfect replacement for Fibre Channel for every possible application. In theory everything you can do with Fibre Channel, can also be done with iSCSI (and InfiniBand and who knows what else,) but the problem here is not theory, it’s the reality of how people use these protocols as well as the actual products and solutions that the storage industry has for customers who are building Data Centers and storage area networks today.The question to answer is not so much whether Fibre Channel is dead or alive today, but what is going to be its future?And the future of Fibre Channel is not solely related to the future of storage, but it is really a bigger and broader issue of how transport technologies in the Data Center will evolve. It’s a given that three protocol stacks are too many for a room as big (or as small) as a Data Center. Nonetheless, neither Ethernet nor Fibre Channel nor InfiniBand in their current incarnations can seriously represent a unified transport option for the Data Center.My opinion is that Data Center transports will converge and at that point Fibre Channel will probably be dead as well as Ethernet and InfiniBand (at least in the way we know them today) and something new will replace them and enable a simpler and more efficient connectivity of devices within the Data Center. But this is not something that happens overnight and it will probably be an evolution and not a revolution. Fibre Channel will have to agonize for quite a long time and it will not die of sudden death. As of today, I can’t see any clear signs of illness, yet.In a nutshell, death is unavoidable for protocols and technologies as well as for anything else we know, but I am not convinced that the Fibre Channel killer will be iSCSI. So the right question to answer should probably be: “Is the Fibre Channel killer born, yet?”

In an effort to keep conversations fresh, Cisco Blogs closes comments after 60 days. Please visit the Cisco Blogs hub page for the latest content.


  1. I have to say that I do not see how iSCSI can be the FC killer. I guess I also have to ask, Why would someone want to kill something that works that well and that reliably?""A few reasons and thoughts for the above-1) iSCSI requires TCP for guaranteed delivery. This will in the end either consume a lot of host CPU cycles on the host and target or it will require TCP Offload Engines that offset and capital expense reduction you thought you were going to get by not implementing FC.2) Ethernet networks like to drop frames. Actually its rather philosophical- we have spent 100s to 1000s of man-hours trying to ensure we dropped the right traffic efficiently and at the right place in the network. Drop frames as a signaling mechanism to higher layer protocols telling them to back off. Suffice to say drop the wrong frame (tail drop on a latency sensistive message anyone?) and you have a problem.3) The day I can see 100-200km synchronous replication of storage arrays across a 10Gb Ethernet network on a high0-volume OLTP database using iSCSI without TOEs I may change my opinion, but to date I do not see how that can be accomplished.As far as why 'kill FC' (I hate the use of that word... but we did not start it eh?) I have to say that until a true converged I/O comes that has the security, simplicity, and price points of Ethernet, the reliability of FC, and probably has some IB features as well I do not think there is a serious contender for anything replacing mainstream deployments of FC.dg"

  2. Marc-I personally think there are some very good applications for iSCSI- net-boot and backup as some good examples.However, I am not sure I agree that iSCSI is the right approach for synchronous backup. Ethernet networks are inherently lossey. That's how they signal congestion to TCP. If I lost the wrong frame it is possible to have a multi-second retrans time. That will certainly disrupt if not completely melt-down a synchronous replication scheme. If the answer to this is 'deploy another completely separate network and try to engineer it such that it cannot drop a frame' I would refer anyone back to the posting we did on 'The Fallacy of Wire-Rate Switching' since it is about impossible to design a network without inherent congestion management and avoidance mechanisms to never drop a frame, and there is no way to tell from packet inspection that any given frame is not the tail of a TCP session. Thus tail drops can and will happen and are rather messy when they do.If and when data link technologies evolve to give us the best of both worlds provided today by Ethernet and FibreChannel (simplicity, cost, and broad appeal for Ethernet, and reliable transfer and best-use of cost-effective optics for FC) I think we will not see a need for the TCP transport layer when running block-based storage over an Ethernet network. Basically, when the underlying datalink can be reliable there is no need for transport layer protocols to add 'yet another' level of reliability. dg

  3. Again, I think its important to compare apples to apples and frame the discussion for storage more realistically than comparing an average Fibre Channel network to an average Ethernet network. If you implement your Ethernet storage (iSCSI) network similar to Fibre Channel networks where there are minimal hops between servers and storage, the occurrence of dropped frames is significantly reduced to the point of being meaningless. Even if you use an existing Ethernet network and connect storage through existing 6500 core switches or stackable switches like the 3750, I/O performance is excellent. The rule of thumb is simple - design an iSCSI SAN to minimize hops. The discussion of TOE requirements benefits from being grounded in application experience. For instance, Microsoft has figured out how to make very efficient iSCSI drivers. If you look at their ESRP data, software iSCSI drivers perform extremely well. Moreover, incredibly powerful and inexpensive multi core servers like the Intel Woodcrest platform run have no problem running iSCSI software drivers. Therefore, the assumption that TOE is required for iSCSI is not realistic. The majority of iSCSI implementations in operation today do not use TOE. These include some extremley high performing transaction databases as well as large enterprise email implementations. To address the last point in the preceeding comment. If I was using synchronous replication for a high volume transaction database, I would sure as heck implement TOE. C'mon TOE doesn't cost very much - about the same as FC, so that's a wash. In this implementation the cost of the adapters - FC or TOE - is mouse-nuts compared to overall captial and operating costs of the network and storage equipment. I don't believe iSCSI is going to kill Fibre Channel. However, I am fairly sensitive to the inverse of that, where FC fans want to bury iSCSI as being not ready for enterprise applications and data centers. Is there a middle ground here somewhere?

  4. Dante, I know you know, but its worth clarifying for others that read your blog: Fibre Channel and Ethernet are transports. Serial SCSI protocols such as FCP and iSCSI do their work at the application layer. Its not the iSCSI application protocol that threatens Fibre Channel, but all the other technologies, talents and tricks that have been invented over the years to make Ethernet/TCP/IP networks manageable, reliable and affordable. A protocol is not the killer, its all the elatives"" that the protocol has and iSCSI has a big family.Which leads to a question. If not Ethernet as the unifying transport in the data center's future, what then? I recently read about CEE as a new standards initiative with the goal of making Ethernet the unifying transport. Maybe that will be some type of new super-Ethernet with certain capabilities that are not backwards compatible with legacy hardware - but still capable of supporting legacy Ethernet networks. It might even be possible to tunnel FC and IB for the sake of compatibility, but tunneling is never quite as clean as native support. I'm interested in hearing more of your thoughts about what you think the future transport of the data center will be. Public speculation only, of course."