Cisco Blogs

Deutsche Banks Talks About Data Center Efficiency

May 17, 2010 - 4 Comments

Submitted by Dave Trowbridge, staff writer for News@Cisco and guest blogger for the Data Center blog.

How much waste is there in your data center? The numbers may surprise you.

The confluence of hard economic times and growing environmental awareness has given a big boost to green data center implementations. More and more companies are making the pleasant discovery that IT sustainability and IT energy efficiency are two blades of the same scissors, that cutting emissions and cutting costs naturally go together, and that if you do one right, you get the other almost automatically.

That doesn’t mean that green data center design and implementation are easy. One of the biggest stumbling blocks is likely to come right at the beginning—justifying the expenditures necessary for a large-scale effort. This can seem like a game of percentages: A bit more efficiency here, a bit more utilization there—how do you get it to all add up to a green light for your green data center?

The answer is that those percentages don’t add up—they multiply! This is what Deutsche Bank IT found when it did the math on data center efficiency as part of its “Eight Commitments to Eco-Efficient IT” in support of the bank’s global sustainability efforts. I recently sat down with Andrew Stokes, Chief Infrastructure Architect at Deutsche Bank, to discuss those commitments and his group’s best practices for green data centers. Here, we’ll drill down into the math behind data center efficiency as an illustration of just how much you may have to gain, and how easy it may be to justify your green data center efforts.


The Three Faces of Data Center Efficiency

As Stokes notes in his News@Cisco Q&A, the fundamental goal driving data center energy efficiency is increasing the amount of business work you get out of each watt consumed. To reach this goal, you need to consider three parameters whose interplay determines overall data center efficiency—data center infrastructure efficiency, relative hardware power efficiency and hardware utilization efficiency.

Stokes uses this equation to figure the overall efficiency of the data center:

(% Data Center Infrastructure Efficiency) x (% Relative Hardware Power Efficiency) x (% Hardware Utilization Efficiency) = % Data Center Eco-Efficiency

“The actual metric for us comes out at somewhere less than 10 percent,” says Stokes. In other words, there’s the theoretical potential to see a tenfold increase in data center utilization! However, he added, that “the absolute value itself is not really of too much significance, because there are many reasons why it would be impossible to achieve 100 percent.”

The goal is much like trying to travel at the speed of light: You can get arbitrarily close with exponentially increasing amounts of effort, but you will never reach it. “For us, the important message is to look at the directional trend of this metric, to commit ourselves to actions that drive the trend in the right direction,” Stokes explains. “We know that this is a multi-year journey, and we see possibilities for a 4x to 8x improvement if we remain focused on this for a whole technology refresh cycle.” To get a sense for the potential improvements possible, let’s take a closer look at each factor in that equation.

Data Center Infrastructure Efficiency (DCiE)

DCiE simply expresses how much of the energy flowing into a data center actually makes it to the computing hardware. This metric comes from the Green Grid, a global consortium of IT companies and professionals seeking to improve energy efficiency in data centers. The Green Grid expresses the overall facility efficiency using either of two green grid metrics: Power Usage Efficiency (PUE) or its reciprocal, Data Center Infrastructure Efficiency (DCiE).

DCiE is defined as IT Equipment Power divided by Total Facility Power, expressed as a percentage, which makes it ideal for the Eco-Efficiency equation. This can be complicated to measure when a data center is part of a facility that houses other functions, and even more so if cooling technologies are incorporated into the IT equipment itself, as is increasingly common. Still, this is perhaps the easiest factor to calculate, at least to estimate with a good degree of accuracy per facility.

Lawrence Berkeley National Labs benchmarked 22 data centers in 2006 and found that the PUE ranged from 1.6 to 3.0 (62.5% to 33.3% efficiency expressed as DCiE). According to the Green Grid white paper on PUE and DCiE, a properly designed data center should be able to attain a DCiE of between 50 and 62.5 percent. Deutsche Bank is pushing the envelope, and has set an initial goal of 55 percent for existing data centers and 75 percent for new ones, and is making good progress towards achieving these goals.

There is some debate amongst professionals about whether the use of PUE and DCiE can drive unintentional consequences. For example, buying servers with more efficient power supplies can actually reduce the overall DCiE efficiency. When blogger James Hamilton wrote about these measurements, Stokes commented that Deutsch Bank “welcomes any steps by key IT vendors to publish the information required to calculate Hamilton’s True PUE (tPUE) and True DCiE (tDCiE). As soon as it becomes practical for us to calculate a ‘tDCiE’ metric, we will use it immediately in our overall DC Eco-Efficiency calculations.”

Relative Hardware Power Efficiency (HPEr)

HPEr is more difficult to derive because it involves the amount of computing capability you get out of each watt delivered to your IT equipment. How do you measure that? Deutsche Bank’s solution was to develop a measurement based on a performance benchmark from Ideas International called the Relative Performance Estimate 2 (RPE2). They divide the performance by the average power utilization to derive a Hardware Power Efficiency metric (‘HPE’). Stokes says a company can use any reputable and externally verifiable performance benchmark that relates well to its core business, and he cautions that there are several factors that go into the actual selection of hardware, of which performance is just one.

The challenge, however, is to convert this absolute number into a percentage, so that it can integrate into the overall equation for DC Eco-Efficiency. Deutsche Bank solved this problem by designating a reference server as the benchmark for ideal efficiency. For Deutsche Bank, this reference server model has to be x86 architecture (so it is suitable for running the vast majority of the Bank’s applications). It has to be a standard purchase item on their purchasing catalog. And it has to have the highest HPE value.

With this in place, Deutsche could then create a Relative Hardware Power Efficiency measure (HPEr) as {Server HPE} / {Benchmark Server HPE} * 100%.

To explain the implications of this number, if you could wave a magic wand and replace all your servers overnight with the benchmark servers, you would create an immediate increase in computing capability, or an immediate reduction in power consumption, or ideally BOTH of these things. Now of course there are many reasons why this cannot be achieved (e.g. all those non-x86 servers, network devices and storage arrays), but the metric is directionally useful in identifying opportunities for improving hardware efficiency, and it is intuitive for technicians and executives alike.

“If we filled our data center with these most-efficient servers,” says Stokes, “we’d more than double our ‘DC Eco-efficiency’ from this part of the equation, even against our current reference server definition.”

There is one further issue with this metric. As technology improves, the reference server absolute metric needs to change, which then affects the HPEr values for all the facilities. As Stokes says, “technology doesn’t stand still. We need to commit to a continual refresh of technology to drive the maximum compute capability from the available power.” Deutsche Bank resets their reference server definition an annual basis.

Hardware Utilization Efficiency (HUE)

Finally, HUE is simply the percentage of CPU utilization of a server. Here Deutsche Bank has set a target of 55 percent for existing servers and 75 percent for new ones. As Stokes admits, “we have some way to go before we can achieve these targets, but we are really focused making an impact here as well.”

However, overall utilization is only half the story; although hardware utilization is a simple thing to measure, it’s not something that lends itself to simple optimization. The problem is that there are typically two kinds of application workloads in a data center, and what’s optimal utilization for one is sub-optimal for the other.

The first kind of workload is throughput-sensitive. Applications in this category at Deutsche Bank include things like accounting and regulatory reporting. The emphasis here is on executing the required processing transactions in a given period, without worrying about the response time of any one transaction. Such applications are not sensitive to latency, and servers hosting them can be pushed to higher utilization without ill effects. “For throughput-sensitive applications,” says Stokes, “we’re aiming for 75 percent utilization, and pushing it higher where it makes sense. We want to stay short of 100 percent though, because we always want to have some spare capacity. We also know that as utilization nears 100 percent, bad things will happen (e.g. context thrashing and excessive system faulting), so we always want to stay clear of those problems.”

The second kind of workload is response-time sensitive. At Deutsche Bank, this includes a variety of trading and real-time applications, where quick response is critical. Such applications are very sensitive to latency, which rises with utilization, so servers hosting them cannot be utilized to the same extent without degrading performance.

“For servers hosting these applications,” says Stokes, “we aim for what looks like a very low utilization of the boxes but actually is optimized for a different dimension—for example, the 10 minutes a day that those applications need to deliver millisecond-level responses to market events.”

For this workload type, the real challenge is to optimize the utilization while being highly sensitive to the desired response time. It’s a fine balance, and one that Deutsche Bank is constantly tweaking as transactional volumes and processing demands change.

Future Opportunities for Development

The DC Eco-Efficiency calculations described here are all infrastructure focused. Industry observers often note that this ignores the impact that can come from application algorithm efficiency improvements, application componentization and re-usage measuring efficiency in business terms. Deutsche Bank is looking at these aspects too. Stokes comments: “We have started with a practical set of measurements from the infrastructure side, but of course we are looking at these other areas of application and business efficiency as well. We recognize that any efficiency changes at the application level act as a further multiplier factor on top of the work we are already doing, so we are fully supporting these efforts.”

As Deutsche Bank has found out – the quest for Data Center Eco-Efficiency has multiplier benefits throughout the organization.

In an effort to keep conversations fresh, Cisco Blogs closes comments after 60 days. Please visit the Cisco Blogs hub page for the latest content.


  1. Great article . A lot of our customers are taking that 1st step towards convergence. Virtualize, automate, standardize, consolidate, by linking these initiatives together, they start to transform aging data center environments into one that’s geared for growth

  2. Well thought out, and very intelligent post.

  3. Great post on data center efficiency, thanks.

  4. Good information,specially the HUE.