Cisco Blogs


Cisco Blog > Data Center and Cloud

Power, Pickups and Polar Bears

May 5, 2008
at 12:00 pm PST

Dave Ohara recently posted on his Green Data Center Blog about the efforts of network vendors to help the greening of the data center. I am still not sure we are having the right conversation around this topic, but at least we are having the conversation.Let me give you an example. Let’s compare a Ford Focus and a Ford F-150 pickup. If we wanted to be”green”, looking at the brochures for both vehicles, the fuel economy of the Focus would be a no-brainer (35 hwy vs. 20 hwy for the F-150). But, what if we wanted to actually”do” something with our vehicle? Since I was working on the yard today, let’s say we wanted to haul 2 yards of mulch. Our F-150, with 55 cubic feet of hauling capacity can do this in one trip, while the Focus, with slightly under 14 cubic feet of cargo capacity would take four trips. So, even with the 75% better gas milage, the Focus is not automatically the right choice. The point is a simplistic one dimensional measure is not really all that meaningful. This is why a city bus can get 4 mpg and still be central a”green” strategy. The other point is that we have multiple types vehicles for a reason--so if I were a landscaper, I may drive my F-150 during the week and drive my Focus when I go tooling in Napa for the weekend. The goal is to derive the maximum value for the energy consumed, regardless of if its vehicles or data center switches.The topic of relevant metrics brings us back to something Dave mentions in his blog: Nortel’s claims that their 45xxT switch consumes 56% less power than an”equivalent” Cisco Catalyst 3750G switch (for the sake of argument, I’ll work with the comparisons that Nortel picked). There is some confusion in the Nortel positioning paper, since the switch model and power consumption noted in the text is different than the model and power consumption noted in the table the text refers to, but I digress. The salient point is that these power values are for two switches plugged-in and idling with no connections--the assertion is that if there is such a discrepancy at idle, imagine what happens when we actually forward packets. Before I go on, a note to those of you with data centers: if you have switches in your data center that are plugged in and not doing anything, please unplug them now--it will help you with power/cooling and the polar bears will thank you too. Now, for the rest of us with data center switches that forward packets, it might be useful to see what a switch does under load.So, earlier this year, at our request, Miercom did an analysis of a number of switches under load to see what kind of power consumption you might actually encounter in your data center. It turns out the Cisco Catalyst 3750G consumes about 140-150W (depending on packet size and choice of copper or optical uplinks) when those uplinks are driven at 100%. This is inline with the 160W maximum draw noted on the data sheet. What does the Nortel 45xxT do under load? That information is not published anywhere I could find, but perhaps we can make an approximation based on a couple of data points we do have. First, the data sheet for the Nortel 4548GT notes a maximum draw of 150W, so unless Nortel is into really over-engineering their power supplies, that is probably a fair indicator of real-world draw at max load. Second, Miercom did test the Nortel 5510 as part of the testing mentioned earlier and they found that switch drew about 125-130W at 100% uplink load against a maximum draw of 135W in the Nortel 5510 data sheet, so there does not seem to be any revolutionary power management technology at work. Third, the notes from Miercom indicate between a nominal 5% load and a 100% load, the power draw grew proportionally regardless of vendor tested. So, I think it is reasonable to deduce that the Nortel 45xxT is in the same neighborhood as the Cisco Catalyst 3750G when used to actually forward packets. Sidebar: Miercom just announced that the Cisco Catalyst 3750-E, 3560-E, 3750, 3560, and 2960 Series Switches are the first products certified in their “œCertified Green” certification program.Later in the same position paper, Nortel asserts their ERS 8610 offers energy savings of 60% over a”6500 equivalent”. There is not a lot of detail to the comparison, although there is a footnote that states”Unless noted, all product comparisons are based on vendor published maximum power ratings.”, so this seems a comparison of power supplies, not actual draw. Again, this becomes a Focus vs F-150 comparison--what exactly are we comparing and in what context--I could dig into this but I think you get the point.I guess the moral of the story here is, when it comes to energy efficiency, there is, sadly, no one magic metric to measure goodness. Its a design function like anything else in the data center and its a matter of doing research and balancing the design parameters. Our goal is, simply, to help you make an informed decision. To that end, we have the Power Calculator to give you configuration-specific load numbers for our equipment. We then take it a step further with our Data Center Assurance Program tool on our data center design best practices website, which includes power consumption information as part of its guidance. Check them both out (you will need a cisco.com account).

In an effort to keep conversations fresh, Cisco Blogs closes comments after 60 days. Please visit the Cisco Blogs hub page for the latest content.

13 Comments.


  1. Omar, I did some checking. Nortel does not post maximum power supply ratings on its web site.. I wonder why. Also the 6500 series can do PoE and the 8600 does not.. so smaller power non PoE supplies = not valid comparison. Same as your Ford F150 and focus comparison. And for price, they use full retail for cisco and nortels discounted price. This is not apples to apples. I think after the switches are loaded, you will find this magic energy tax dissapears. This campaign sure does seem deceptive. Again, Why will Nortel not post their maximim power consumption on their website..

       0 likes

  2. Correction, I did find some”" Nortel equipment actually posted it’s max power ratings. Hmm. It looks like the equivilant system in Cisco has about the same ratings.. No magic here. Also the Nortel IP phones have a disclaimer that no one else uses. “”Power does not include line/cable loss|. To be valid, they should include this. I suspect all the other vendors have taken this into account. I would also like Mercom to test the new Nortel switches for temperature sensitivity. I suspect they have written some code to slow down/turn off cooling fans at idle. Cisco turns them on to meet military specs for operating in harsh hot environments on start up. If they are saving power by limiting cooling, the switches will not have the same capabilities to operate in some of the telecom closets throughout the world. It does not pay to skimp on cooling. We have seen many pre-mature failures due to this. Again.. apples to oranges.”

       0 likes

  3. Omar, Would be glad to comment, I have many years experience with different systems and this looks like a challenge worth solving. I think we have several tools at our disposal to analyze useful work. We can get server statistics and a breakdown of IP traffic workload by application through various automated statistics gathering programs like vitalsuite and others. If the goal is to cut power consumption, there are now many best practices for data center layouts which contain things like, no more raised floors, perforated doors, proper clustering of servers and using access switches close to them to reduce cable clutter that in turn blocks heat dissapation. I have also found that how you cool things can have a dramatic effect. If you pass cold air over a hot exit stream, it takes almost an infinate amount more cooling capacity to get the desired cooling level. So layout is critical. We can easily measure voltage, current and heat load as stated in the white paper. This combined with automated matrix could produce some real statistics. Changing gears, one other thing I have noticed, it looks like GigE does take a little more power to support, but I do think the speed is worth it as the storage media has less run time to send out the data stream. That would be a challange to analyze the cost/benefit. So the age old question of performance over efficiency will have to be studied. More later.Best Regards, E.

       0 likes

  4. Omar,Another general comment, Data centers allow for people to do many things via web interface verses having to talk to a live customer service agent (CSA). I think any energy required to run a data center is peanuts compared with the cost and support of a CSA not to mention the productivity and time savings. We should keep that in prospective when discussing if data centers take to much energy. Also we are reducing the number of paper bills, payments, tranfers, questions, etc. The savings is huge, for the small price of a data center. Just a thought. Best Regards,E.

       0 likes

  5. Omar Sultan

    E:I think bring up a good point–there is more info in terms of best practices then there was even a couple of years ago, and I think that helps. However, I think there is also significant oppty in terms of tools to help customers understand the energy cost of a given workload, which is first step to figuring out how best to handle it.This next part may be heretical, but I don’t think the goal should be to cut consumption. As you point out in the second post, folks continue to invest in IT because, at the macro level, there is a net benefit to the business. In light of that, I think the goal is not consumption, but efficiency–if I am going to consume a watt, what is the most work I can get out of it. I think we should assume consumption will grow, but we should strive to flatten that growth curve as much as possible.

       0 likes

  6. Omar,This blog is interesting. You gave an example of a Ford Focus versus a Ford F-150. The problem is this analogy uses the same manufacturer although different vehicles with different capabilities meeting different demands. If you were to compare a Catalyst 3750 with a 6509, this would be more applicable.However, keeping with your example, let’s assume a 6509 is a Ford F-150. In 2 x wheel drive, the 6509 line cards are capable of layer 2 switching. All routing and QoS functions are centrally performed by the Supervisor module potentially causing bottlenecks and delayed responses. In order to perform routing and QoS on each line card, a DFC daughter card is required per line card module (per Cisco documentation). This would require the Ford F-150 to have another engine per wheel to enable 4 x wheel drive. Using a 9 x slot 6509 comparison, if the F-150 had 7 x wheels (assuming 2 x wheels are used for redundant Supervisor modules/main engine), this would be 7 x additional engines that each consume gas.The problem doesn’t stop at the additional gas used. If I upgraded my Ford F-150 from a stock 4.2-liter V6 (Sup32) to a 4.6-liter V8 (Sup256), I have to purchase all new engines for all 7 x tires. Same holds true when upgrading from a 4.6-liter V8 (Sup256) to a 4.8-liter V8 (Sup720).If I put my F-150 into 4 x wheel drive and wanted to cross mud, that would require a specific set of tires (DFC modules). However, next time I may want to cross rocks, sand or other terrain. That would require a different set of tires (DFC, DFC2, DFC3A, DFC3B, DFC3BXL, DFC3C, DFC3XL). An additional problem exists in that not all tires (DFC modules) work with the main engine (Supervisor module).Even the main engine (Supevisor module) has a second engine (PFC module daughter card) that consumes gas and needs to be upgraded when adding fog lamps, rear window defrosters, etc.. The main engine’s second engine upgrades can cause the air conditioning or radio to stop working, depending on the tire engine in use, and require tire engine upgrades.I think if the CxO’s had more understanding of technology, the 6509 would have not seen the success it has had with or without power consumption savings. Unfortunately, most buying decisions occur at lower levels where Cisco certification rule and commands a higher pay grade mostly to understand the complexities of deploying Cisco products vs. the actual technology itself.Nortel, Foundry, Extreme and others don’t share the same design flaw exacerbated by the importance of keeping Cisco IOS alive, through CPU driven approaches and proprietary ASIC design. As society itself continues to become more astute technologically, we’ll find ourselves adopting more open standards based technology that breaks the barriers of proprietary vendor architectures and relagates vendor proprietary approaches to features”" that make the vendor stand out among others in an open medium.Networking should be viewed as an intelligent highway. When the car driven on that highway is dictated by the manufacturer of the highway, progress for anyone other than the manufacturer of the highway is lost. Take a look into SONA vs. SOA and you’ll see a glimpse into where the future of the supposed “”open”" highway is being defined.”

       0 likes

  7. Ah, and the Cisco marketing machine continues to churn. There are a lot of companies that make great product, I don’t know why this one can’t just admit when they are wrong. It is like being in a court room citing only favorable and couter case evidence and the defendant is mute.As a 3rd party end customer after working with Cisco, Foundry, Extreme, Nortel and many others, just because it is teal doesn’t mean it is the best. Nortel has a great product. Cisco products are heavily marketed and prolonged in their architecture to maximize profits. The reality is, there is other documentation out there supporting the Nortel claim that has conveniently not made its way into this blog. Imagine that! If truth be known, I didn’t believe such claims until I tested a Nortel 4500 against a Cisco 3750. Oh, and with traffic and various IP phones, PCs, and etc. Guess what? Nortel comsumes less at idle, with traffic, and in any situation. Moreover, you can’t really compare the 4500 to the 3750. The 4500 has WAY more performance. This is not a blog about technology or product, it is mere damage control. As a side note, I learned today that Cisco has filed a law suit against Nortel based upon their ‘Green Campaign’. Hum. Didn’t they file an injunction against another organization so they wouldn’t expose the Clean Access holes. Is this the same company that told me sun spots was causing my 7500s to reboot? Yeap. Cisco makes good products. The only problem is there are better products out there and Cisco refuses to give some credit where it is due. Educate yourself! Blogs like this are damage control. Nothing else.

       0 likes

  8. GMG733 – Can you be more specific about your test. What kind of phones did you use, how many PoE ports did you load up.. Oh you forgot to tell everyone that the 4500 cannot use all the ports at the PoE standards rate.. I wonder why.. oh.. they do not have as much power available (maybe thats why they use less). What temperature did you run your test at. What equipment did you use for measurement. Sounds like a Nortel marketing campaign in your note. I would be nice if people would actually use real test conditions. And there are no Nortel products in the whole world that do not have problems. Sounds like Nortel is doing damage control on CNN every day. Keep the facts and stop putting your bias in the blogs.

       0 likes

  9. Bias? I did a search on Miercom Reports and found this:Extreme Networks – 0 Reports Foundry – 1 Report Nortel – 6 Reports Cisco – 40 ReportsI would really like to know if Cisco provided the test plan when they
    equested”" the analysis from Miercom or if Miercom developed the plan.To be considered “”non-bias”", Miercom would need to perform the same testing on other vendors too.Does a company have to pay Miercom for this type of analysis?Just Curious….”

       0 likes

  10. After reading this blog and I decided to check out the Miercom site for more information. Does a company
    equest”" and “”pay”" for an analysis/test or do these folks do it with the best interest of the networking community in mind?(quote from Omar’s post – “”…So, earlier this year, at our request, Miercom did an analysis of a number of switches.”")I’m not sure how they do business but I found the following “”metrics”" from the Miercom website (Reports)to be quite interesting. I did a search for reports over the last 6 years and here is what I found.Foundry ( 1 Report)Extreme Networks ( 0 Reports)Nortel ( 6 Reports)Cisco (40 Reports)WOW! In today’s political environment, that could be viewed as “”slanted”" at best.When Cisco requested the analysis, did they dictate the testing or did Miercom create the test plan? I understand this is a Cisco blog, but I would really like to see the same Miercom analysis for other vendors. This would lend more credibility to the claims in this blog…just my opinion.”

       0 likes

  11. Omar Sultan

    I will leave it Miercom to establish their credentials and discuss their business model, but a quick review of their website shows that they offer a number of testing services.There is always a question with using third part testing. I think best you can strive for is to have transparency of who is doing the funding and how the testing was done. Miercom is up front about who engaged them in their reports and I also noted our engagement in my blog entry. Their testing methodology and certification requirements are their own.We engaged third party testing because there is no agreed upon metric and test methodology, so individual vendor claims tend to be rather pointless. We wanted to see how our switches did compared to a peer group when all the switches we subjected to a consistent test methodology that modeled real-world usage. The outcome of this effort is the Miercom report. The methodology itself is documented in the report, so anyone is free to validate the test results.Ultimately, its up to you to decide if Miercom and this report are credible sources of information for you.

       0 likes

  12. Omar Sultan

    Well, you bring up some interesting points, but I think we might be at the point where this car analogy has run out of gas, so to speak.I guess, I do not buy into the multiple engine analogy. On said F-150, the same engine will support both 2×4 and 4×4 configurations, the same way a switch is designed to work on within a given performance envelope. After all, the 2×4 versus 4×4 choice is not about power or speed (i.e. the 4×4 does not go faster) but other performance parameters (i.e. traction). You could make the same analogy about the level of over-subscription on a line card or I/O module.Either way, it underscores my original point, gas mileage by itself is not a useful measure, and may even be misleading. Evaluating efficiency–the work done for fuel consumed–is a more useful guide.Regards,Omar

       0 likes

  13. Omar Sultan

    E:Thanks for your follow-ups. You make some good points on the consistency of testing parameters and seem to have some good experience on this front. Unfortunately, while power and cooling is front and center, it is hard for many of our customers to make informed decisions right now. The first challenge is that this is a new skill set for many network folks–power efficiency is a new dimension and part of the challenge is understanding what is truly relevant. It is like walking into and electronics store and looking at two amps–one rated at 75W RMS and the other at 100W RMS and $100 more–is the extra 25W worth the extra $100? Probably not, but I am sure that many a customer is up-sold on a daily basis. Much like Total Harmonic Distortion or Signal to Noise Ratio, they are impressive sounding metrics and easy to grasp, but largely useless in choosing an audio component these days.The other challenge is a dearth of standardized information. At this point, the best we can offer is third-party testing, simply because it ensures a common testing methodology across the equipment tested. On a positive front, I think we are seeing some movement on this front. There is Meircom’s Green cert and there is some good work being advocated by the Green Grid (http://www.thegreengrid.org/gg_content/White_Paper_13_-_Framework_for_Data_Center_Energy_Productivity5.9.08.pdf) on establishing meaningful metrics. I am not sure if these specific efforts will pan out, but I think it is heading in the right direction.I’d be curious to get your thoughts on the Green Grid white paper.Regards,Omar

       0 likes