« November 2007 | Main | February 2008 »

January 31, 2008

Let's Talk Bandwidth

You know, it's time I get back to my geeky roots here some and talk about tings that I find very exciting- backplanes, fabrics, forwarding engines.... BANDWiDTH! It's one of the many reasons I love blogging, because we can talk about things that matter in IT that most don't care about: Big Pipes!

On the Nexus 7000 we use numbers like 15 Terabits, 230Gb per slot, and so one... let's take one of these and extrapolate HOW we get there. I think this may help in drawing comparisons to other architectures out there.

230Gb per slot, and how we get there...

From the I/O module to the Fabric Module there is a series of thick copper traces and high density connectors. Each pair of copper traces is today clocked at 3.125Gbps. Each fabric channel is comprised of 16 pairs of copper for 16x3.125Gbps or 50Gbps half-duplex and 25Gbps of full-duplex bandwidth per fabric channel. (each pair of Cu at 3.125Gbps is unidirectional) We have to encode on the wire and we use a 24b/26b encoding scheme that yields just north of 23Gbps of real-world bandwidth per fabric channel.

Each switch fabric chip has 26 fabric channels. 2 fabric channels connect from each switch fabric ASIC to each line card in the 10-slot chassis. 2 Fabric channels per slot @ 23Gb each is 46Gb per slot per Fabric Module.

The Nexus 7000 can sustain up to five fabric modules. So with five switch fabric modules @ 46Gbps each we have 230Gb per slot. Now I do want to clarify that this is 230Gb IN and 230Gb OUT concurrently from every slot in the system when five fabric modules are present. Bandwidth scales up and down linearly with the addition or removal of fabric modules. You need a minimum of one, and max of five.

How do we assure we use all that bandwidth effectively?

Firstly the Nexus 7000 has a virtual output queue system with fabric arbitration, or for short and Arbitrated VOQ system. VOQ is where we have a queue on each ingress I/O module for every egress port on the system. We have 1024 queues on every I/O module, one for each potential 10GbE interface.

Packets go through L2 and L3 lookup on the I/O modules forwarding engine with then replies with a Fabric Port of Exit header. This FPOE header is used by the fab ric to forward a packet to the right destination. It is also used to to identify which VOQ to put the packet into.

If we have lots of 64b frames in the VOQ we can concatenate them so we only put one FPOE header on a larger block of 64b frames. If we have jumbo frames we segment them into smaller sizes that are more optimal for the fabric and deliver more deterministic latency, especially under load.

Each VOQ can only send data across the fabric when the receiving I/O module has the capability of receiving the data and serializing it onto the egress interface. This is extremely beneficial as it pushes all congestion to the source, aligns all of the larger buffering into one place, and is one of the key technical attributed that allows us to ensure a lossless fabric architecture. The fabric arbiter also ensures that we use all fabric modules bandwidth as efficiently as possible.

15Tb? Well if you have been building a spreadsheet or following along with a calculator just know that the same system applies to the 18-slot chassis with 16 I/O module slots. And that the 3.125Gbps number we discussed per pair of Cu unidirectionally? Well we know for certain we can do over 2x that today, if not even higher...

Posted by Douglas Gourlay at 01:27 PM Permalink | Comments (1) | TrackBacks (0)

January 29, 2008

Cisco Nexus 7000: "Changing Data Centers Forever"

In this video, Doug Gourlay, Cisco Senior Director of Data Center Solution Group, talks about our new Cisco Nexus 7000 Series Switch - purpose built for the Data Center of the future. It is designed to transform data center operations, making our customers experience in the data center more efficient, resilient and lasting. Also, check out the Cisco Nexus 7000 website where you can explore the features and elements of the Nexus 7000 Series in our 3-D Interactive Model and view other videos introducing the Nexus 7000 family.

Posted by Cisco PR at 10:45 AM Permalink | Comments (0) | TrackBacks (0)

Why a new OS?

I read an interesting article questioning why we created NX-OS. It's a fair question so one I figured I'd pu together a few quick thoughts on....

NX-OS is not yet another Operating System. NX-OS has as its core Cisco SAN-OS. It is a modular, multi-process, multi-threaded Operating System that has full Stateful Process Restart and Zero-Service-Disruption Upgrades.

SAN- yes
LAN- yes
IP- Yes
IPv6- Yes

We really see NX-OS as our strategic operating system in the data center and actively envision multiple data center products converging into this single OS over the next year or two.

So a SAN administrator will find every feature they are used to, a LAN administrator familiar with the Catalyst line can pick it up and be productive in a few minutes.

We also created Virtual Device Contexts- this allows multiple administrators to operate a single Nexus concurrently. One Admin could restart OSPF and the other Admin's OSPF would not be impacted at all. Complete process segmentation and separation, basically running server virtualization on the network control plane.

This lets you run a Lab-Net on top of production, model configuration changes, then cut them into production. It lets Service Providers offering hosting services allow some level of customer self-management for the top-tier customers that mandate it and also reduce failure domains. And lastly a SAN administrator and a LAN administrator can co-habitate without messing each other up.

Posted by Douglas Gourlay at 10:39 AM Permalink | Comments (4) | TrackBacks (0)

January 27, 2008

Nexus- Defined

Nexus: The Core, or Center.

After spending 12% of my life working with the finest engineers, marketing professionals, and product management today we are announcing the Cisco Nexus Family. Led by the flagship Cisco Nexus 7000 Series- the first Data Center Class Switching Platform.

What do we mean by data center class? Start with carrier class, then build up. Make a system that is designed to operate inline with a customer's operational best practices. Make a system that works the way you do.

Why Nexus? A nexus is a connecting point, or the core depending which line you read in the above URL. To us at Cisco Nexus is the foundation for building virtualized data centers, the foundation for unifying the data center fabric, the convergence point of SAN, LAN, and High Performance Computing.

I'll probably be writing a lot about the Cisco Nexus 7000 over the next several weeks - we'll discuss the new OS - NX-OS and how it enables data center virtualization, we'll chat about unified fabrics and what makes them the single most important innovation in Ethernet since switching was pioneered 15 years ago, and we'll delve into Cisco TrustSec: scalable security that simplifies operations.

For now click through to http://www.cisco.com/go/nexus and root around a little.

Posted by Douglas Gourlay at 10:17 PM Permalink | Comments (2) | TrackBacks (2)

January 25, 2008

The Power of “And”

Post by: Omar Sultan, CCIE
Solution Manager – Data Center Switching
CMO – Data Center Solutions

The other day, I was reading through a message board on another website was reminded, how our industry tends to look at things in an adversarial light. A new box is released and we need to compare it to what’s out there and choose sides in a “red switch/blue switch” showdown. In this case, I was reading through a spirited discussion about FCoE vs. iSCSI. In this particular case, once the dust settles, I think folks will figure out that there is room for both technologies in their data center, much as folks figured out it a few years ago that they can blend SAN and NAS as part of their storage strategy—neither technology is perfect, but they can compliment each other to provide a stronger overall solution.

Since the Super Bowl is coming up, perhaps a football analogy is apt: if you draft a Heisman Trophy winning running back, you don’t bench your pro-bowl wide receiver, instead, you explore what opportunities you can create by taking advantage of both assets—use the running game to set up up the pass. This is the power of “and”.

If you are a follower of this blog, you know Doug has hinted that some cool things are in store. As these announcements unfold, I invite you to set aside the natural tendency to compare and instead focus on the opportunities these new products and technologies open up for you.

Posted by Omar Sultan at 01:33 PM Permalink | Comments (0) | TrackBacks (0)

January 23, 2008

Application Delivery Networks!

This week Cisco introduced the Application Delivery Network to the press and analyst corps as well as our key European customers at Networkers 2008 in Barcelona. This architectural approach to application delivery showcases how the network can sense changes in application requirements and dynamically adjust resource allocation to meet changes in demands. It also clearly highlighted how Cisco can understand applications at a message-passing level and link this deep insight to load balancing at the flow level and end-to-end transit at the packet level.

Possibly even more impactful is the tremendous support garnered from top application partners- for each key application we have tested, documented, and profiled how to enhance the applications performance through a variety of integrated network-based tools from WAN optimization to TCP Offload in the ACE Family.

We also introduced the ACE Appliance a 1-2Gb Application Delivery Controller that has the best price-performance in its class and the WAAS Mobile Client – extending the reach of WAN Optimization to the mobile worker and telecommuter.

What do we need to build next?

Posted by Douglas Gourlay at 10:13 PM Permalink | Comments (0) | TrackBacks (0)

Cisco pushes network to deliver apps

So Networkers Barcelona is in full swing, and Cisco and partners are making their announcements.

What does Cisco have to say about the hot "application delivery" market? Some pretty interesting things, actually.

Like adding app smarts to your routers and switches, and then tying them to your app switches and WAN optim, actually gives you an end 2 end solution (we call it an "application delivery network"). And more bang for your application buck.

And then there's the new products we're announcing -- like new virtualized app switching in a sub-$16K USD appliance form factor (the ACE 4710 appliance). And WAN optimization for the remote user, via desktop software client -- called WAAS Mobile.

Here's what one European attending publication had to say about Cisco at the event:

http://www.news.com/Cisco-speeds-up-mobile-workers-application-access/2100-1012_3-6227140.html?tag=item

But most interesting is what the software vendors themselves say about what Cisco can do for their customers' apps + networks:

http://newsroom.cisco.com/dlls/2008/ekits/exec_commentary.pdf

Stay tuned for more to come. Or share your experience with application delivery products and deployments with a posting back...

Posted by Mark Weiner at 08:47 AM Permalink | Comments (0) | TrackBacks (0)

January 22, 2008

Wise Words From The Bard...

I saw an announcement today in the Data Center space that reminds me well of some famous words by the inimitable William Shakespeare... but we'll get to those in a few moments.

It looks like companies one solely focused on storage networking are now following in our footsteps yet rather than providing vision, strategy, and execution the are providing vibrant PowerPoint showcasing an intent to start becoming a network player. I applaud this deciison and welcome them to the Ethernet party, it started 15+ years ago in switching but new market entrants are always welcome. Certainly pioneering innovations in Ethernet like switching itself, lossless forwarding, VLANs, FibreChannel over Ethernet, EtherChannel, etc are all some protocols that Cisco has innovated that may be of use- but don't forget RFC1149, it's a must-have for new players in this space... and apropos.

Tune in next week for some real announcements in the Data Center, ones that build a resilient, efficient, and lasting infrastructure with an architectural approach that is backed by furious execution.

As for The Bard... I always loved this quote from MacBeth-

"...a poor player that struts and frets his hour upon the stage and then is heard of no more. It is a tale... ...full of sound and fury, signifiying nothing." ---William Shakespeare

Posted by Douglas Gourlay at 10:25 AM Permalink | Comments (0) | TrackBacks (0)

January 06, 2008

Watch This Space...

Have you ever run in a race? I used to run some in high school - I was never very good at it, but it was fun to compete. I always got this nervous energy and butterflies in my stomach coming up to the starting line. It's that build-up of excitement right before something monumental to you at that time, something exciting, something visceral.

I'm having that same feeling right now.

I've been here for about 10 years now- and have spent the last four working in the data center space on some really ground-breaking technologies in both the Application Networking area and the core infrastructure space. It's amazing to think that I have spent a bit over 10% of my life working on one project, but that's what keeps me going.

I titled this entry 'Watch This Space' because over the next several weeks and months there are going to be significant announcements in the data center from Cisco. Not just new boxes or features, but technologies that will transform the way networks are built, change the way IT operates, and set a new bar for availability, security, and manageability. That's why I am so excited right now... the race is about to start.

Watch this space.

Posted by Douglas Gourlay at 09:40 PM Permalink | Comments (2) | TrackBacks (0)

January 04, 2008

Virtual Switching in the Data Center

Heard of Virtual Switching System on the Catalyst 6500 yet? This is something that was recently tested by David Newman over at Network World, and is linked here.

I am quite excited about VSS- it is a technology that fundamentally changes the way we build data centers, period. Sure the performance is nice, but the real benefit is twofold:

1) No Spanning Tree Loops are manually created in order to effect redundant paths. No blocked links, all active, and the failover times get cut by up to 95%. As much as it upsets my friends in IEEE 802.1 I would rather design Spanning Tree OUT of my network than have it IN my network with artificial loops.

2) Reduced IP routing complexity. If every Aggregation pair of switches represents themselves to the core as one device this reduces EIGRP and/or OSPF routing topology complexity by 50%. Less devices means a smaller routing table, simpler topology, and still blazing fast failover speeds, that are actually quicker than what we had with just L3 Routing.

I wanted to extend a special thanks to the small and tight-knit group of Engineers and PM's who made this concept a reality, thanks team. This is going to change the way people design networks, simplifying them, and making them more resilient than ever.

Posted by Douglas Gourlay at 11:47 AM Permalink | Comments (0) | TrackBacks (0)

A New Years Challenge...

Happy New Year to everyone! Am sitting here in not-so-sunny Northern California watching storm after storm hit and it had me wondering about Disaster Recovery a bit. I love the fact that networks connect things and that distance = options where DR is concerned.

How far apart should data centers be? How far do you usually deploy them? We all understand the issues related to SCSI timeouts and synchronous replication of storage that gates teh maximum distance for synchronous replication to the 100-200km range depending on your application.

Is it easible that companies will tier applications and dynamically move their primary execution location based on pending known natural disasters?

Is it feasible that a brownout scenario or possibility of rolling-blackouts on the power grid would be a reasonable leading indicator to trigger the movement of compute and storage resources from one location to another?

What are the other challenges that you are encountering? Are their organizational barriers to this or is DR so ingrained now that it is purely an execution issue?

Posted by Douglas Gourlay at 11:38 AM Permalink | Comments (0) | TrackBacks (0)