Cisco Blogs

Cisco Blog > Data Center

Towards an Industry Standard for Benchmarking Big Data Workloads

Industry standard benchmarks have played, and continue to play, a crucial role in the advancement of the computing industry. Demands for them have existed since buyers were first confronted with the choice between purchasing one system over another. Over the years, industry standard benchmarks have proven critical to both buyers and vendors: buyers use benchmark results when evaluating new systems in terms of performance, price/performance and energy efficiency, while vendors use benchmarks to demonstrate competitiveness of their products and to monitor release-to-release progress of their products under development [1]. Historically we have seen that industry standard benchmarks enable healthy competition that results in product improvements and the evolution of brand new technologies.

Over the past quarter-century, industry standard bodies like the Transaction Processing Performance Council (TPC) and the Standard Performance Evaluation Corporation (SPEC) have developed several industry standards for performance benchmarking, which have been a significant driving force behind the development of faster, less expensive, and/or more energy efficient system configurations.

The world has been in the midst of an extraordinary information explosion over the past decade, punctuated by rapid growth in the use of the Internet and the number of connected devices worldwide. Today, we’re seeing a rate of change faster than at any point throughout history, and both enterprise application data and machine generated data, known as Big Data, continue to grow exponentially, challenging industry experts and researchers to develop new innovative techniques to evaluate and benchmark hardware and software technologies and products.

I am co-chairing a workshop with my distinguished colleagues Chaitanya Baru, Meikel Poess, Milind Bhandarkar, Tilmann Rabl and others entitled Workshop on Big Data Benchmarking (WBDB 2012) [2], supported by the National Science Foundation ( This is a crucial initial step towards the development of an industry standard benchmark for providing objective measures of the effectiveness of hardware and software systems dealing with Big Data. Several industry experts and researchers have been invited to present and debate their vision on benchmarking big data platforms.

A report from this workshop will be presented at the just-announced 4th International Conference on Performance Evaluation Benchmarking (TPCTC 2012) [3], organized by the TPC, which will be collocated with the 38th International Conference of Very Large Data Bases (VLDB 2012), a premier forum for data management and database researchers, vendors and users. With this conference, we encourage industry experts and researchers to submit ideas and methodologies in performance evaluation, measurement and characterization in areas including, but not limited to: big data, cloud computing, business intelligence, energy and space efficiency, hardware and software innovations and lessons learned in practice using TPC and other benchmark workloads [4].

Cisco has been an active member of the TPC since 2010 and the SPEC since 2009.

[1] R. Nambiar, N. Wakou, P. Thawley, A. Masland, M. Lanken, M. Majdalany, F. Carman: Shaping the Landscape of Industry Standard Benchmarks: Contributions of the Transaction Processing Performance Council: Springer 2011
[2] Workshop on Big Data Benchmarking:
[3] TPC Press Release:
[4] TPCTC 2012 Call for Papers:

Tags: , , , ,

Final Thoughts on the Open Networking Summit

So, some closing thoughts on ONS.  I know its a bit late, but hey, when you’re out of the office for a few days, things pile up a bit–overall, I think the ONF folks did a fine job with the event.

As I look back at ONS, I am reminded of one of my favorite IT quotes, courtesy of Bill Gates:

We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten

Long-term, I think SDN or the concepts it represents will certainly have a hand in shaping how we do networking a decade for now–how we get there and what that destination really looks like is a bit less certain.

First, I think we are early enough in the game that the technology is far from unsettled:

  • Most folks are shipping 1.0 code, either literally or figuratively, and I am betting there are unseen technologies in the wings that will help shape things and I am sure folks will find interesting ways to also repurpose existing technology
  • We can pretty much expect some wave of M&A to help shape the vendor and technology landscape
  • As I have noted before, there is a lot of dogma about what SDN is right now that is not helpful, but I also believe it will eventually fall by the wayside

Eventually the market will sort this stuff out, and a handful of organizations are in a position to drive their own solutions, but for regular folks, I think there is enough near-term uncertainty here that it will give people pause–both in terms of customer adoption as well as ecosystem investment.

Read More »

Tags: , , ,

Return on Big Data – How Workload Automation Plays a Key Role

A few weeks ago I participated in on a webinar panel
around Big Data and the return on that data.  I was joined by Ivan Chong, EVP from Informatica and Van Baltz, VP and CIO of Station Casinos,, where we discussed Van’s aggressive project to roll out a realtime big data system.  This  deployment , using Informatica Powercenter and the Cisco Tidal Enterprise Scheduler was an impressive project that drove both a new architecture and critical changes to the people, processes and technologies.   Ivan used a great analogy around the exhaust from applications.   Nowadays, IT shops deploy and run many applications that run the real-time business for their companies.  All these applications produce exhaust, namely the data after all that work is done.  This data is very valuable.  It can tell a lot about your business, your customers and your execution to meet the corporate financials.

Cisco’s Tidal Enterprise Scheduler is a master at moving this data around and processing it so that the business IT users have the data they need to run a real-time business.

Read More »

Tags: , ,

So what do we *not* do in Cisco Data Center Services?

Recently I blogged on the rise of UCS and my own perspectives joining Cisco Data Center Services around the launch of Cisco UCS back in March 2009.  I then posed a quick poll on the Cisco Data Center Facebook page, with a number of options, asking which of these options did we in Cisco Data Center Services *not* offer to our customers today. Thanks to all who took the time to answer the poll.  So let’s look at the summary of our services I presented in my previous blog (diagram below), and let’s discuss what you said via the poll.

Cisco Data Center Services Portfolio Evolution 2008-2012

Cisco Data Center Services Portfolio Evolution 2008-2012

Read More »

Tags: , , , , , , ,

Begin your Journey to the Cloud with Cisco’s Cloud Starter Edition

Please be aware that this product is no longer sold.

As Jason Schroedl  announced, Cisco’s Intelligent Automation Solutions Business Unit, in conjunction with the Unified Computing System has just announced a solution for customers of UCS and vCenter that want a Cloud Automation system that can perform both Physical and Virtual server provisioning.  It is called the starter edition for a reason.  We find that many customers are not sure what they want from their cloud and are looking for a great place to start.   This is not what I call the “starship enterprise” of clouds.  It is the first step that a company will take on their cloud journey.

See my previous blog for some key concepts of success cloud deployments: and on my cloud owner manifesto for successful cloud builders: .

Let’s look at typical cloud deployments.

Read More »

Tags: , , , , , , , , , , , , ,