Data Center & Server Switching Category Archives
May 01, 2008
Cisco & Ovum Analyst Discuss Data Center Trends at Interop
Doug Gourlay, senior director, Data Center Solutions at Cisco and Mark Seery, vice president of Switching & Routing at Ovum RHK, discuss data center trends at Interop 2008. These include consolidation, Fiber Channel over Ethernet (FCoE), SFP+, Twinax cabling, and new products.
Cisco Data Center Solutions
Posted by John Murphy at 09:02 AM Permalink | Comments (0) | TrackBacks (0)
March 26, 2008
Data Center Virtualization R Us
Allan Leinwand had a provocative blog post on gigaom.com the other day about the imminent arrival of a blade server for the Nexus. Doug ably explained why we would not do such a thing, so I am not going to revisit that, but I do want to explore the question Allan posed at the end of his entry:
Who do you think can provide those resources more effectively -– a blade server manufacturer using virtualization with networking added to the system or a data networking manufacturer adding blade servers and virtualization?
Not wanting to let Allan have all the fun, let me take a run at answering this question of who is better positioned to support data center virtualization, and, by extension, server virtualization. If you think about it, data center virtualization is about pulling down walls and blowing up silos--its about giving an application the most appropriate resources at a given moment based on the priorities and needs of the business. Underlying this is the idea of making data center infrastructure appear as homogenous pools of resources--you are able to supply processor cycles to your application or storage to your application without being so concerned about the specifics of the underlying physical asset. Cisco DNA is rooted in concept of delivering ubiquitous access to data center resources. With an AGS+ in my data center I could access a collection of disparate hosts without concerning myself with how those particular systems connected to the network. A few years, our SNA integration strategy (DLSw anyone?) opened up the value companies had locked up in their mainframes to anyone with an IP stack. Fast forward to today and one of the benefits of a unified fabric is providing ubiquitous SAN access to any server. See the common theme here? Expect that trend to continue as we continue to execute on Data Center 3.0.
Contrast this with the blade environment, which is, by nature, proprietary and closed. What ever innovation is delivered, no matter how cool, is walled inside a vendor specific, form factor specific silo. Multiple blade vendors? Too bad, you are locked-in. Application requirements dictate rack servers, tower servers, or mainframes? Sorry, can’t help you. You end up in an architectural cattle chute or you end up trying to manage disparate virtualization schemes across your data center. Neither seems all that appealing to me.
Finally, when it comes to virtualization experience, the leading blade server vendors today are not delivering server virtualization, VMware, Xen, and Microsoft are. On the other hand, we are delivering network resource virtualization with VDCs in the Nexus, VLANs across the Catalyst family, VSANs in the MDS family, L4-7 service virtualization in products like ACE and the Firewall Service Module, VBS switch virtualization in our next-gen blade switches, heck, even the new ASR is running KVM.
Granted I am a little biased these days--although I did start out life as a VAX and System V sysadmin--but getting back to Allan’s original question, while we are not quite there yet, I think Cisco has the shorter distance to travel here....but I am guessing there are some of you out there that might have a different viewpoint...? ;)
Posted by Omar Sultan at 12:16 AM Permalink | Comments (0) | TrackBacks (0)
February 01, 2008
WHY YOU’LL WANT THIS SWITCH, PART 1
As I Google “Cisco Nexus” and look through the articles, it is a rare article that doesn’t mention the “15 terabit” number within the first couple of sentences. I can certainly understand this because its a sexy number and its an easy way to quantify the engineering goodness baked into this platform. But, to be honest, if, after spending all this time, money and effort on a new platform, all we could talk about was speeds and feeds, it would be a little sad.
Ironically, the reason I think folks will want a Nexus are a couple of areas that have not gotten much press so far. The first of these “secret weapons” is NX-OS, which has actually been a source of some trepidation. So, first off, “new” is a bit of a misnomer. What we did was start with the proverbial clean sheet and design an OS explicitly for the data center. We had the latitude and R&D capability to cherry-pick from the Cisco technology pantry and build from scratch as appropriate. So, we chose SAN-OS, which has been running for over 4 years in some of the most demanding enterprise data centers out there, as the foundation. Likewise, our layer 3 code, from our Procket acquisition, has been deployed with several major service providers for a number of years. Finally, our layer 2 code was written with the same team that made the Catalyst so successful.
As you can see, we took a very modular approach to building the OS, which is one of the reasons we can successfully integrate these components. In fact, I believe that the fact that NX-OS is a extremely granular and modular OS built atop a secured Linux kernel will appeal to the vast majority of customers. If offers a number of advantages in terms of flexibility, stability and extensibility that other folks will be hard pressed to meet. And, by the way, we wrapped all this in an IOS-like CLI so ramp-up time for staff is minimal. OK, I’m a bit biased, but I think we have hit the mark in terms of giving you operational and functional consistency with the existing Catalyst family, while also giving you an OS that will grow with you the next decade or so.
But we’re not done yet. The framework of NX-OS also allows us to break new ground in terms features and functionality. One of the coolest new features are Virtual Device Contexts (yes, the engineers named this one). The NX-OS brings hypervisor-like functionality to the Nexus, so you can deploy multiple virtual switches on a single physical platform. This lets you support multiple environments on a single switch, say dev and production, or different lines of business with different business polices. VDCs also allow you to segment your environment to keep your security and compliance folks happy. In short, VDCs allow you to drive higher return out of your investment through more efficient utilization and greater flexibility.
That’s about it for now—you can go to http://www.cisco.com/go/nxos for more details. Next time, I’ll dig into exactly what “zero service disruption” means.
Posted by Omar Sultan at 12:22 PM Permalink | Comments (2) | TrackBacks (0)
January 29, 2008
Cisco Nexus 7000: "Changing Data Centers Forever"
In this video, Doug Gourlay, Cisco Senior Director of Data Center Solution Group, talks about our new Cisco Nexus 7000 Series Switch - purpose built for the Data Center of the future. It is designed to transform data center operations, making our customers experience in the data center more efficient, resilient and lasting. Also, check out the Cisco Nexus 7000 website where you can explore the features and elements of the Nexus 7000 Series in our 3-D Interactive Model and view other videos introducing the Nexus 7000 family.
Posted by Cisco PR at 10:45 AM Permalink | Comments (0) | TrackBacks (0)
January 25, 2008
The Power of “And”
Post by: Omar Sultan, CCIE
Solution Manager – Data Center Switching
CMO – Data Center Solutions
The other day, I was reading through a message board on another website was reminded, how our industry tends to look at things in an adversarial light. A new box is released and we need to compare it to what’s out there and choose sides in a “red switch/blue switch” showdown. In this case, I was reading through a spirited discussion about FCoE vs. iSCSI. In this particular case, once the dust settles, I think folks will figure out that there is room for both technologies in their data center, much as folks figured out it a few years ago that they can blend SAN and NAS as part of their storage strategy—neither technology is perfect, but they can compliment each other to provide a stronger overall solution.
Since the Super Bowl is coming up, perhaps a football analogy is apt: if you draft a Heisman Trophy winning running back, you don’t bench your pro-bowl wide receiver, instead, you explore what opportunities you can create by taking advantage of both assets—use the running game to set up up the pass. This is the power of “and”.
If you are a follower of this blog, you know Doug has hinted that some cool things are in store. As these announcements unfold, I invite you to set aside the natural tendency to compare and instead focus on the opportunities these new products and technologies open up for you.
Posted by Omar Sultan at 01:33 PM Permalink | Comments (0) | TrackBacks (0)
November 19, 2007
So Google is building a 10GbE switch?
Today I ran across an article about Google building its own 10GbE switch:
http://gigaom.com/2007/11/18/google-making-its-own-10gig-switches/
It's no secret that Google has been building its own servers for their data centers for some time and is now one of AMD's top customers. It probably makes a lot of sense for Google to extend this to the network infrastructure and start building it's own 10GbE switches for server access. With as many servers as Google is installing, anything it can do to take cost out of its infrastructure and improve efficiencies in space and power will certainly help it scale.
Now server design is fairly straightforward and understood and most of the components have been commoditized. But can the same be said for network switches?
Cisco and other network equipment vendors have built up years of experience in network switch design and optimized the architecture for performance and reliability. With Google entering this space, there are several questions that could be asked.
Can Google hope to replicate this design expertise in just a few years?
If Google is succesful and building a low-cost, high-performance 10GbE switch, what effect will it have on the networking industry?
Should networking engineers start looking at Google for new and exciting opportunities in switch design?
What other technologies should Google look to invest in?
Will this buy vs. build argument sway other companies into considering this approach?
What is certain is that Google, because of its size and cash warchest, can move into any industry at will and potentially become a disruptive force to an incumbent player.
Posted by Deepak Munjal at 04:43 PM Permalink | Comments (3) | TrackBacks (0)
July 26, 2007
Data Center 3.0?
Posted by:
Omar Sultan, CCIE
Solution Manager – Data Center Switching
CMO – Data Center Solutions
You may be wondering if there is anything to this “Data Center 3.0” thing beyond some clever marketing folks earning their paychecks. Well, clever marketing folks aside, the name was very explicitly chosen, so lets deconstruct it a bit.
First of all, this is not LAN 3.0 or SAN 3.0, but Data Center 3.0. The point is that, looking ahead, you can expect a more holistic approach to Cisco data center solutions. The best example of this is the Data Center Assurance Program—its not just product or feature testing, but full-fledged system integration testing. VFrame Data Center is an other example of a solution designed with a pan-data center perspective
Second, if you look at where the industry is heading and more importantly, how our customers looking to transform their businesses, we feel we are on the cusp of another inflection point. To successfully ride this transition is going to take fresh thinking and new solutions. Granted, we need to build upon what is currently successful, but we also need to step outside the box in terms of products and solutions, hence, we have 3.0.
Data Center 3.0 encompasses this mix of both the evolutionary and the revolutionary. For example, we have taken our existing storage networking solutions and added new form factors for design flexibility. We have also added new capabilities such as Storage Media Encryption and the Cisco N-Port Virtualizer because they address pressing customer concerns such as regulatory compliance and simplification of server infrastructure.
On the other hand, VFrame Data Center inaugurates a whole new category of tools. Actually, if you take VFrame Data Center, Cisco Smart Call Home and the Cisco Data Center Assurance Program together, you can see on of the goals of Data Center 3.0 is to not just allow you to do more or do better, but to do easier—simplify operations.
Finally, Data Center 3.0 is about keeping pace with the increasing complexity and scope of customer’s application environments. At Networkers, for example, we introduced a trusted WAN optimization solution to offer secure WAN acceleration and more precise application performance management. Along the same lines, the Cisco ACE XML Gateway allows an increased ability to monitor and manage XML traffic. These solutions are elements of a broader Data Center 3.0 strategy to increase the application fluency of the network, which, we believe, will support further customer innovation.
I think that final point is the real value of Data Center 3.0—it’s all about giving our customers the tools to innovate. If you watch the video clip of Jayshree Ullal on this blog, her final comment is worth remembering: the things we launched at Networkers this week are simply the beginning—they establish a vision and a vector and give you a hint of some of the cool things you can look forward to in the coming months.
Posted by Cisco PR at 03:46 PM Permalink | Comments (3) | TrackBacks (0)
April 27, 2007
File Networking on Friday
Am sitting in my office today recovering a bit from a head cold induced by a cruise ship in the Caribbean. These ships, while making an amenable focus on the eradication of germs and diseases by rampant hand washing and sanitizing stations, could still be known as the '100,000-ton Petrie Dishes of the Sea'. But I digress - this is a blog about data center technologies and today we are going to incite a bit of a conversation about file networking.
There is a new term getting bandied about -- one of the 'File Area Network'. This sounds marginally contrived since these systems run on top of traditional LAN technologies, but let's not debate too much the relative merits of weak taxonomy :) These file management systems and file virtualization systems allow for a single global namespace in the enterprise and probably more importantly, abstract the file naming and external volume association from the storage medium and device itself.
This allows the IT administrator to start provisioning storage resources based on policy rather than user preference and user selection. For example an enterprise could choose to have a file share that represented itself as the 'Z:' drive to an end-user. However, whatever files the end user copied to the Z: drive would actually be stored across 3 or 4 different storage targets based on metadata and user information relevant to the actual files being stored. For instance all MP3 and MP4 files could be stored on a low-cost medium, while anything with the words 'Company Confidential' coming from an end-user in the Finance department could be stored on a Tier-1 storage target while ensuring the data was encrypted in flight and at rest.
This is just a simple example of how file virtualization technologies could be used - there are many more. As you may have heard Cisco announced the acquisition of NeoPath a leader in this space and as we move forward with the productization of this technology into network transport platforms we are trying to get a feel for how you would use these capabilities and make them part of your operational procedures.
So how would you use file virtualization? What do you see as the major opportunity for the technology? What platforms should we integrate it into? What business problems will it help you solve?
Thank you
dg
Posted by Douglas Gourlay at 04:52 PM Permalink | Comments (1) | TrackBacks (0)
February 05, 2007
Ethernet over Barbed Wire, Arcnet, 100MB Token Ring, 100Base-VGAnylan and iSCSI …
I do about 5 executive briefing center visits a week discussing Cisco’s data center strategy. I have a beautiful slide deck highlighting the advantages of Cisco’s end-to-end data center strategy; the builds and animation are fantastic, the graphics superb, and of course my oratory is reminiscent of Lincoln’s Gettysburg address . However, inevitably I get asked a question on Cisco’s commitment to the data center and in particular., for some odd reason, our commitment to FibreChannel.
While I believe that most of this may come from competitive FUD, some of it probably arises from Cisco’s initial foray and positioning into the storage I/O market. You see, before Cisco entered into the Fibre Channel switching business, Cisco had a pretty strong position on iSCSI taking over the storage I/O business. This was not quite the Voice will be Free proclamation but certainly we had our own dogma about the direction of storage I/O. We purchased an ip storage company, co-authored the iSCSI specification and set forth to conquer the storage networking market.
As we all know, the adoption curve of iSCSI in the storage networking industry has had a much shallower ramp than was initially predicted and since then Cisco has successfully entered into the Fibrechannel storage networking market. What we learned from this experience is to rely on a Cisco strength -- to be transport agnostic -- and let the market place and our customers drive our technology focus and investment. Cisco made its name based upon providing solutions that interconnected networks over disparate technologies. This brings me to the headline of this blog. At various times Cisco has deployed or demonstrated a variety of technologies that didn’t quite pan out from a customer or market deployment model. We demonstrated Ethernet over Barbed wire (great for war zones or cattle ranches), sold lots of products with Arcnet, contemplated 100Mb Token Ring and, my favorite, developed router interfaces for 100Base-VGAnylan.
So when a customer asks me about our commitment to Fibrechannel, I say that we are as committed to Fibrechannel as long as the storage market and customers demand networked Fibrechannel connectivity. This philosophy of being transport agnostic is reflected in our current networking portfolio in the Data Center, where Cisco offers Ethernet, FibreChannel and Infiniband solutions. Ultimately the market will decide which technology (or future technology) becomes dominant.
Now can someone tell me whether I should buy a Blue Ray or HD DVD player?
Ed Chapman
Posted by Ed Chapman at 07:00 AM Permalink | Comments (0) | TrackBacks (0)
January 29, 2007
The Fallacy of Wire-Rate Switching in the Data Center
Some days I think that I have heard the question, ‘is your switch wire-rate?’ so many times that if I only had a nickel...
Wire rate switching simply means that if I have a gigabit port I can put a gigabit of traffic through it to some other port without dropping any packets. Multiply this by the number of ports in the device and you have a 32-port 10Gb wire-rate switch for instance.
The majority of Cisco’s switching and routing portfolio is not wire-rate. Now this is interesting given that a large portion of the market acquires these products and should be indicative of the fact that many people really do not care so much about this term. Let’s instead focus on what I like to call the ‘Fallacy of Wire Rate Switching’.
First let’s look at how wire-rate is achieved and what it means. To perform at wire-rate there are two important performance metrics- one is forwarding engine performance (measured in packets per second or packet lookups per second), the second is usually referred to as backplane capacity.
Forwarding Engine performance- This on a Catalyst 6500 is the performance of the PFC/DFC or what we sometimes call the EARL. EARL7 (PFC3) performs IPv4 lookups at ~48-50mpps. There are some features that when implemented on a system reduce the forwarding engine capacity because those features require a packet to either go through the forwarding engine ASICs twice or more. These are features like IPv6, some MPLS operations, GRE, etc.
Backplane Capacity- This is a simplified term that specifically would mean the amount of bandwidth delivered to each slot of a modular system via a backplane or midplane. However, this term usually implies the entire forwarding path from an ingress port, through a series of Port ASICs, Fabric ASICs, Switch fabric ASICs, and then back out the egress port. It is entirely possible to have a backplane that is capable of wire-rate or even overspeed but have a blocking scenario somewhere in the packet data path between the ingress physical port and the backplane.
So when someone claims, “Full Wire Rate Switching at all frame sizes’ they are implying that for each port there is a non-blocking data path through the switch port to port and that the forwarding engine has the capacity to do packet lookups, ACLs, QoS remarking, etc without slowing down the packet forwarding rates.
So let’s talk about reality for a second.
1) Every vendor remotely existing in switching designs products around four metrics- Bigger, Faster, Denser, Cheaper.
2) Most vendors stretch a bit when they claim wire-rate. They tend to singularly optimize on three frame sizes to the exclusion of realistic traffic patterns. 64-byte frames are always run so they can claim a high packet per second number; 1514-byte frames are run as the non-Jumbo Ethernet Frame size max, and 9k Jumbo’s are run if the product supports Jumbo Frames (most do now). Try a 65-byte frame for fun and we have found that many products put more packets in the proverbial bit bucket than out the egress port... For a ‘realistic’ frame sizing use IMIX. It’s around a 300-byte packet size average. Or for even more fun and games intersperse 90% Jumbo Frames with 10% 64 byte frames - on a non-arbitrated fabric this will cause massive issues almost all the time... (it is also a semi-realistic traffic pattern to see large frames and acknowledgements concurrently)
3) To build a wire-rate switch the vendor must make some sacrifices. This all comes down to physics and trade-offs. If you want to build a wire-rate switch you will make a few decisions to trim down things. Commonly what we hear is the design teams saying- “We built a wire-rate N Gigabit backplane and when you build a WIRE RATE SWITCH YOU DON’T NEED BUFFERS’. This is critical! It is arguably one of the biggest mistakes vendors make.
Let’s analyze this for a second before getting off the tangent ;)
VendorX makes a 200-port wire rate switch. To build this they trim the off board memory systems down a good clip and move the memory on-board to save ASIC pin count and allow ‘wider’ lookups. This improves performance and allows them to achieve wire-rate forwarding for most frame sizes.
We can safely assume that a vendor can build a wire-rate switch. But have you ever seen a wire-rate network? A network with true 1:1 any to any bandwidth with no oversubscription. This means a network with no EtherChannel (which can cause hash collisions, thus link contention) no Equal Cost Multipath (same as EtherChannel), and a design that never oversubscribes the uplink. Realistically this farcical network would have to have no Spanning Tree somehow since that would block any looped paths and reduce the effective bandwidth.
Even if someone can build a ‘wire rate network’ (I hope the previous paragraph dissuaded you of the notion) have you ever seen an application optimized perfectly for a wire-rate any to any network? Pretty much every app I can think of has some fan-out. Someone hits a web server which talks to a first-tier app server which fans out to 8 other app servers which poll N number of data bases, reverse the path and deliver the application/content/page to the client. Significant fan-out.
Any time we look we can realistically see that SOMEWHERE in the network there is the rational possibility of oversubscription and we need buffers to handle this. When vendors build their so-called wire-rate switches they sacrifice the buffers on the altar of wire-rate!
Why do buffers matter? Quite simple actually- if 2 ports are talking for the briefest instant to one port at 10Gb/s each buffers allow the packets to interleave long enough so that both ports get served without sending a packet spiraling to a lonely death. (although with small buffers that death might not be so lonely and might be more lemming-esque)
If the network has a propensity to fan out, use EtherChannel, have uplinks that are smaller than the aggregate of downlinks, use Routing with Equal Cost Multipathing, or has pretty much any real world application on it you will have a fan-out and thus contention for a port from 2 or more other interfaces. These small-buffered wire-rate switches perform abysmally in these types of real-world scenarios.
How should switches be designed for real-world applications and data centers?
First off stop being dogmatic about wire-rate. Performance is important but based on the above separate performance from wire-rate. Performance is about effective throughput under real-world conditions.
Adequate and effective buffers designed to absorb contention and congestion and effectively deliver real-world applications is of paramount importance. Intelligent dropping of the right traffic when congestion occurs is also equally important to make most effective use of the buffers that do exist. Until better L2 congestion management systems come into widespread deployment buffers are the best way to deal with this.
The big four metrics of Bigger, Faster, Denser, Cheaper is a limited value proposition that is not sustainable. Let’s face it- the Biggest, Fastest, Densest, Cheapest switch on the market changes about once every 9-12 months.
Data center switching systems focus more on features that could carry a taxonomy such as "More Stable, Highly Available, Faster Converging, Operationally Manageable, Feature Rich, Segmented, Secure, Integrated, Systems Level Tested, etc."
Thanks,
dg
Posted by Douglas Gourlay at 02:48 PM Permalink | Comments (3) | TrackBacks (2)
