Architecture of a Platform
In the last post I talked about the importance of designing a platform vs. a box and the benefits associated to that choice. Now I’d like to spend some time going into more of the architectural details that make the MDS box a platform. I’m going to try to keep the discussion as high level as possible, but at the end we are still a technology company, so I hope at least some of you will appreciate this “engineering” discussion.The architecture of an MDS Director can be simplified by listing the main three components involved in the packet switching operations: – the port module (or linecard): the element delivering the forwarding functionalities for each packet received in an ingress port and transmitted to an egress port – the crossbar (either on the supervisor or on the fabric module): the ‘N x N’ matrix providing connectivity between each port in each module – the arbiter (on the supervisor module): the scheduler deciding which port should be using the crossbar to transmit trafficBased on these building blocks, the MDS Director switches deliver a centrally arbitrated, crossbar architecture, with frame forwarding logic distributed in each port module. A packet arrives in an ingress port, a forwarding decision is made by the port module, the arbiter schedules the packet transmission based on congestion, the crossbar switches the packet to the egress port.By having switching (crossbar) and forwarding (port module) functionalities decoupled, the customer can seamlessly upgrade to the next interface speed by simply upgrading the crossbar. The investment in existing common equipment (chassis, supervisors, power supplies, etc) and port (or service) modules is preserved as new higher speed modules can now coexist in the same chassis. Capital expenses are significantly reduced by fully utilizing the open slots in existing chassis; operational costs associated to “rip-and-replace” approaches are completely eliminated. Decoupling functionalities is not the only factor required to achieve such investment protection and flexibility: the protocol and interface speed agnostic nature of the crossbar is the other key element. Such a crossbar can itself switch any packet independently of interfaces’ speed (2-, 4-, 8-, 10-Gbps) and the protocol utilized for the transmission. In fact, the exact same crossbar used in the MDS 9000 Family to carry Fibre Channel traffic is used also in other Cisco Ethernet products! The protocol agnostic aspect of the crossbar is what for example will allow existing MDS to seamlessly switch 10-Gbps Fibre Channel Over Ethernet (FCoE) packets to deliver upon the Unified Fabric vision, as previously discussed.But is this the only way to design a switch? In the storage networking industry, alternative examples inspired by the 3-stage Clos network are actually present. In this type of approach, each element is a device combining both switching and forwarding functionalities, simply meshed with the others via internal Inter Switch Links (ISLs), like it was a network of small switches. Certainly this design has advantages from a development perspective: you simply need to design that one device and you can re-use it across the 3-stages of the high-end Director, as well as in the low-end Fabric Switch version. Time to market is also reduced. But what about the customer? The customer is forced to completely rip and replace the infrastructure every time a new speed is introduced. And what about supporting a new protocol, at new higher speed, considering that that device is not protocol (Fibre Channel based forwarding) nor speed (8-Gbps) agnostic? Will yet another forklift upgrade be required or something weird will have to happen behind the scenes to use that Fibre Channel forwarding or the lower 8-Gbps speed? And in this mesh of devices, where is the central scheduler ensuring the traffic is flowing smoothly by solving congestion situations?